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ACRONYMS  AND  TERMS 

Acronym  or 

Term  Definition 

ADL  Advanced  distributed  learning.  Training,  instruction,  and  assessment  delivered 

over  the  Web. 

BMK  Basic  marksmanship  knowledge.  Selected-response  measure  intended  to  sample 

basic  knowledge  about  USMC  rifle  marksmanship. 

CTSR  Classroom  Test  of  Scientific  Reasoning. 

CWO  Chief  warrant  officer. 

DL  Distributed  learning,  distance  learning.  Used  synonymously  to  refer  to  Web-based 

learning. 

ELR  Entry-level  rifle  marksmanship. 

ESP  Evaluation  of  shooter  positions.  Online  task  intended  to  measure  a  participant's 

knowledge  of  proper  and  improper  shooting  positions. 

GCT  General  Classification  Test. 

KSA  Knowledge,  skills,  and  abilities. 

KD  Known  distance.  Course  of  fire  at  specified  distances  (e.g.,  200  yards). 

LT  Lieutenant. 

Phase  I  Rifle  marksmanship  classroom  training  period,  Wednesday  and  Thursday  of  Week 

1. 

Phase  II  Rifle  marksmanship  live-fire  period,  Monday  through  Thursday  of  Week  2. 

Sustainment-level  rifle  marksmanship. 


SLR 
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DETERMINANTS  OF  RIFLE  MARKSMANSHIP  PERFORMANCE: 
PREDICTING  SHOOTING  PERFORM ANCE  WITH  ADVANCED 
DISTRIBUTED  LEARNING  ASSESSMENTS^ 


Gregory  K.  W.  K.  Chung,  Girlie  C.  Delacruz,  Linda  F.  de  Vries,  Jin-Ok  Kim, 
William  L.  Bewley,  Adriana  A.  de  Souza  e  Silva,  Roxanne  M.  Sylvester, 

and  Eva  L.  Baker 

CRESST/University  of  California,  Los  Angeles 


March  31, 2004 
ABSTRACT 

The  UCLA  National  Center  for  Research  on  Evaluation,  Standards,  and  Student 
Testing  (CRESST)  is  under  contract  to  the  Office  of  Naval  Research  (ONR)  to 
conduct  research  on  assessment  models  and  tools  designed  to  support  Navy  and 
Marine  Corps  distance  learning  (DL).  The  first  such  application  is  in  support  of 
USMC  marksmanship  training.  In  a  series  of  studies  we  examined  the  role  of 
cognitive  and  non-cognitive  variables  in  the  prediction  of  rifle  marksmanship 
performance.  Prior  research  on  predicting  shooting  performance  suggests  a 
deceptively  complex  task  sensitive  to  a  variety  of  variables.  The  stages-of-skill- 
development  model  (Ackerman,  1987, 1992;  Fitts  &  Posner,  1967)  suggests  cognitive 
measures  will  be  most  sensitive  to  individuals  in  the  learning  phase,  and  perceptual- 
motor  measures  most  sensitive  to  individuals  past  the  learning  phase.  The  role  of 
cognitive  variables  (knowledge  of  shooting  in  particular)  is  largely  unexplored 


1  We  would  like  to  thank  the  staff  at  Stone  Bay  WTBN,  especially  Col.  Sheldon,  LtCol.  Harrelson, 
CWO  Conrad,  CWO  Bennett,  SSgt.  Armistead,  Sgt.  Butterbaugh,  Sgt.  Burke,  Sgt.  Behan,  Capt. 
Athanasiadis,  MSgt.  Race,  and  Cpl.  Cabe.  We  would  also  like  to  thank  the  staff  at  Quantico  WTBN, 
especially  MGen.  Jones,  BGen.  Flynn,  Major  Thomas,  Col.  Bourgeois,  Col.  Kerrigan,  CWO 
Pipenhagen,  Major  Bourne,  Capt.  Hasseltine,  SSgt.  Jones,  Sgt.  Greene,  Sgt.  Pritt,  GySgt.  Kyle,  GySgt. 
Witherspoon,  and  SSgt.  Pinheiro.  We  also  thank  all  the  Marines  from  Stone  Bay  and  Quantico  who 
participated  in  these  studies.  Special  thanks  to  Steve  Jones  of  Mitre  Corp.  for  facilitating  the  process. 
We  also  wish  to  thank  the  following  people  from  UCLA/CRESST:  Joanne  Michiuye  for  her  help  with 
the  preparation  of  this  manuscript  and  with  data  collection,  Nicole  Kersting  and  Gale  Stuart  for  their 
help  with  data  analysis,  Cecile  Phan  and  Gary  Dionne  for  their  help  with  data  collection,  and  David 
Brill,  Farzad  Saadat,  Ravi  Sinha,  and  Matthew  Zhang  for  programming  and  technical  support. 
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beyond  examination  of  shooting  performance  across  groups  receiving  different 
training  and  instruction. 

In  a  series  of  studies  we  were  able  to  predict  record-fire  performance  between 
.52  to  .86,  depending  on  the  sample.  Bivariate  correlations  between  various 
measures  and  record-fire  scores  were  obtained  in  the  .2  to  .8  range.  Perceptual- 
motor  measures  —  intended  to  reflect  experience— were  consistently  a  good 
predictor  of  performance.  The  most  recent  record-fire  score  predicted  record-fire 
score  at  the  .3  to  .4  range.  The  best  single  predictor  of  record-fire  score  was  the  firing 
line  experience  survey,  which  yielded  correlation  coefficients  from  .6  to  .8.  Cognitive 
measures  (aptitude  and  knowledge  related  to  marksmanship)  in  less  experienced 
samples  related  to  record-fire  score  in  the  .2  to  .4  range.  No  relationships  between 
record-fire  score  and  knowledge  measures  were  found  in  the  more  experienced 
sample.  Affective  measures  (worry,  anxiety)  predicted  record-fire  scores  in  the  -.3  to 
-.6  range  and  in  general,  for  the  affective  and  firing  line  experience  measures,  state 
measures  had  coefficients  of  higher  magnitude  than  the  trait  versions. 

Overall,  we  have  gathered  evidence  that  in  general  suggests  a  knowledge 
component  to  shooting  performance.  The  results  of  our  studies  point  to  differences 
in  knowledge  of  rifle  marksmanship  between  participants'  pre-classroom  training 
and  post-classroom  training,  between  more  experienced  participants  and  less 
experienced  participants,  between  high  performers  and  low  performers,  and 
between  higher  aptitude  and  lower  aptitude  participants.  Knowledge  measures  can 
predict  record-fire  scores  moderately  in  less  experienced  samples,  and  when 
combined  with  other  variables  within  the  stages-of-skill-processing  framework,  can 
predict  record-fire  scores  as  well  as  scores  from  a  rifle  simulator. 

Rifle  marksmanship  is  a  complex  psychomotor  skill  sensitive  to  variations  in 
the  individual,  equipment,  and  environment.  It  is  unlikely  that  variation  in  the 
equipment  and  environment  can  be  reduced  much,  thus  leaving  the  individual  as 
the  only  area  for  improvement.  Given  that  we  have  found  a  cognitive  component  to 
rifle  marksmanship  performance,  it  may  be  that  improving  a  Marine's  knowledge  of 
rifle  marksmanship  will  have  the  most  cost-effective  payoff.  Early  identification  and 
remediation  could  lead  to  increased  cost  savings  in  travel,  decreased  time  away 
from  the  Marine's  home  unit,  increased  throughput  on  the  firing  line,  increased  time 
coaches  spend  providing  feedback  to  shooters  on  the  firing  line,  and  lower 
ammunition  and  target  costs.  In  addition  to  cost  savings,  early  identification  and 
remediation  could  lead  to  higher  scores  overall  and  fewer  unqualified  Marines. 
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SECTION  I:  OVERVIEW 
Background 

Rifle  marksmanship  is  a  core  value  of  the  Marine  Corps.  The  creed  "Every 
Marine  is  a  Rifleman"  embodies  the  value  Marines  place  on  marksmanship.  Marines 
are  recognized  as  having  the  best  marksmanship  training  and  riflemen  of  the 
uniformed  services  and  the  Marines'  competitive  marksmanship  program  has 
consistently  generated  world-class  shooters  since  its  inception.  Marines  have  to 
undergo  annual  qualification  and  their  performance  accounts  for  part  of  a 
promotional  decision.  Regardless  of  occupation,  whether  their  job  is  infantry,  air 
combat,  or  support,  and  regardless  of  weapons  specialty,  every  Marine^  must 
qualify  on  an  M16A2  rifle  at  least  as  a  marksman  or  it  is  unlikely  he  or  she  will  be 
promoted. 

The  UCLA  National  Center  for  Research  on  Evaluation,  Standards,  and  Student 
Testing  (CRESST)  is  under  contract  to  the  Office  of  Naval  Research  (ONR)  to 
conduct  research  on  assessment  models  and  tools  designed  to  support  Navy  and 
Marine  Corps  distance  learning  (DL).  The  project  is  called  Knowledge,  Models  and 
Tools  to  Improve  the  Effectiveness  of  Naval  Distance  Learning,  or  KMT.  The 
approach  to  conducting  KMT  research  has  been  to  develop  and  test  tools  designed 
to  address  the  assessment  and  training  requirements  posed  by  real  Navy  and 
Marine  Corps  training  applications.  The  first  such  application  is  USMC 
marksmanship  training. 


USMC  Context  of  the  Work 

Ensuring  that  every  Marine  qualifies,  as  well  as  improving  sustainment-level 
training  such  that  every  Marine  qualifies  as  expert,  is  the  concept  behind  program 
HUEY  (Unqualified  [UNQ]  to  Expert),  developed  by  Lt.  Col.  Carl  Shelton  of  the 
Stone  Bay  Weapons  Training  Battalion  (WTBN).  Initially,  the  idea  was  that 
assessment  tools  and  simulations  could  be  used  to  provide  remedial  training  to 
Marines  after  they  failed  to  qualify.  Later  conceptualizations  evolved  to  the  idea  of 
the  potential  of  using  distributed  learning  technology  —  online  assessments  and 
instruction— to  assess  Marines  and  if  needed,  provide  instruction  before  Marines 


2  Some  exemptions  apply  based  on  experience  or  rank,  such  as  Marines  with  20  years  or  more  of 
active  duty.  Colonels  and  above.  Sergeants  Major  or  Master  Gunnery  Sergeants,  and  CWO  4  and  5. 
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reached  the  firing  line  — and  ideally,  before  they  reached  Stone  Bay.  Bewley  et  al. 
(2003)  describe  the  current  marksmanship  training  process  in  more  detail. 

The  potential  for  time  and  cost  savings  is  there;  Early  identification  and 
remediation  could  lead  to  increased  cost  savings  in  travel,  decreased  time  away 
from  the  Marine's  home  unit,  increased  throughput  on  the  firing  line,  increased  time 
coaches  spend  providing  feedback  to  shooters  on  the  firing  line,  and  lower 
ammunition  and  target  costs.  In  addition  to  cost  savings,  early  identification  and 
remediation  could  lead  to  higher  scores  overall,  fewer  UNQs,  and  more  experts. 
However,  the  potential  for  cost  savings  and  enhanced  shooting  performance  is 
contingent  on  the  ability  to  first  identify  potential  UNQs  and  potential  low  shooters. 

This  document  reports  on  a  series  of  studies  conducted  to  examine  the  role  of 
cognitive  and  non-cognitive  variables  in  the  prediction  of  rifle  marksmanship 
performance.  We  begin  with  an  introduction  to  rifle  marksmanship,  then  review  the 
literature  on  predicting  shooting  performance,  and  end  the  section  with  our 
validation  strategy  and  research  questions.  The  next  major  section  reports  on  the 
domain  sampled  and  presents  an  overview  of  all  of  the  study  findings.  The  third 
major  section  reports  on  the  pilot  studies,  and  the  fourth  major  section  reports  on  the 
main  studies. 


Introduction  to  Rifle  Marksmanship 

In  this  section  we  describe  our  conceptualization  of  rifle  marksmanship  as  a 
kind  of  skilled  sport.  We  first  briefly  describe  what  makes  shooting  difficult,  and 
then  discuss  five  variables,  based  on  the  literature,  that  affect  shooting  performance. 

Why  Is  Consistently  Hitting  a  Target  Difficult? 

One  of  the  most  remarkable  achievements  in  USMC  marksmanship  training 
and  weaponry  is  in  developing  a  shooter's  skill  to  routinely  hit  a  19-inch  circular 
area  at  500  yards  in  the  prone  position.  Five  hundred  yards  is  about  1.5  times  farther 
than  the  distance  between  two  people  at  opposite  ends  (lengthwise)  of  the  Los 
Angeles  Coliseum  (see  Figure  1).  What  makes  this  achievement  even  more 
remarkable  is  that  virtually  any  deviation  of  the  rifle  from  the  center  line  will  result 
in  a  miss.  A  rifle  muzzle  deflection  of  1/16  inch  (about  the  thickness  of  a  quarter) 
from  the  center  line  will  result  in  the  bullet  strike  being  off  by  about  five  inches  at 
100  yards  and  over  2  feet  at  500  yards. 
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Figure  1.  Example  of  the  USMC  qualification  distance  in  the 
slow-fire  prone  position;  approximately  1.5  stadium  lengths  of 
the  Los  Angeles  Coliseum. 

Adding  to  this  complexity  are  uncontrollable  factors  such  as  wind  velocity, 
gravity,  and  ammunition  ballistics.  For  example,  a  10-mph  breeze  (enough  to  raise 
dust  and  loose  paper)  displaces  a  round  about  2  feet  over  500  yards.  Gravity  alone 
results  in  the  round  dropping  20  inches  over  300  yards.  Variations  in  the  amount  of 
propellant  across  bullets  result  in  10-inch  shot  groups  at  300  yards  for  skilled 
shooters  (U.S.  Army,  1989). 


These  examples  do  not  take  into  account  factors  associated  with  the  shooter— 
perhaps  the  most  variable  component.  Normal  breathing  in  the  standing  position 
can  displace  the  rifle  muzzle  1/2  inch  from  inhale  to  exhale,  while  changes  due  to 
the  heart  pulse  can  also  displace  the  muzzle  a  fraction  of  an  inch.  If  a  shooter's  sight 
alignment  is  off  by  a  fraction  of  an  inch,  the  shooter  is  unlikely  to  hit  the  target. 
Fatigue  decreases  results  by  causing  shaking,  wobble,  or  other  instabilities;  flinching 
or  bucking  due  to  recoil  or  reaction  to  the  report  causes  the  shooter  to  jerk  the  rifle, 
as  does  pulling  or  yanking  the  trigger.  Exacerbating  position  instability  is  the 
emotional  state  of  the  shooter  —  anxiety  and  other  factors  increase  the  heart  and 
breathing  rates.  Finally,  the  recoil  from  the  rifle  can  cause  the  muzzle  to  rise  about  20 
milliradians  (Torre,  Maxey,  &  Piper,  1987).  Figure  2  shows  how  minute  changes  of 
the  rifle  affect  the  bullet  strike  under  ideal  conditions.  For  example,  moving  the 
muzzle  1/8  of  an  inch  results  in  the  bullet  strike  being  off-center  by  nearly  2  feet  at 
200  yards. 
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Figure  2,  Estimated  effects  of  muzzle  displacement  on  bullet  strike  at  different  distances  under  ideal 
conditions.  Actual  displacement  is  greater. 


Thus,  accurately  and  consistently  hitting  a  target  is  a  complex  interaction  of 
factors  immediately  before,  during,  and  immediately  after  the  round  goes  off: 
establishing  and  maintaining  sight  alignment  on  the  target,  maintaining  postural 
steadiness,  not  disturbing  the  rifle  while  squeezing  the  trigger,  and  adjusting  for 
environmental  effects.  Virtually  any  deviation  from  a  motionless  position  will  result 
in  a  miss,  virtually  any  deviation  from  a  perfectly  aligned  and  sighted  rifle  will 
result  in  a  miss,  and  lack  of  compensation  for  wind,  distance,  and  other 
environmental  factors  will  result  in  a  miss. 

Further,  accurately  predicting  shooting  performance  may  be  difficult  given  the 
low  reliability  of  record-fire  scores  (Schendel,  Morey,  Granier,  &  Hall,  1983).  The 
relationships  between  successive  trials  have  been  found  to  range  from  no 
relationship  (Marcus  &  Hughes,  1979)  to  moderate  correlations  in  the  .5  range 
(Vielhaber  &  Lauterbach,  1966),  to  high  correlations  (MacCaslin  &  McGuigan,  1956) 
in  the  mid-  to  high  .80s. 
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Prior  Rifle  Marksmanship  Research 

There  has  been  little  effort  to  understand  rifle  shooting  as  a  complex  skill  as 
opposed  to  other  sports  (c.f.,  baseball  [French,  Nevett,  Spurgeon,  Graham,  &  Rink, 
1996;  French,  Spurgeon,  &  Nevett,  1995;  Nevett  &  French,  1997],  tennis  [McPherson 
&  French,  1991;  McPherson,  1999a,  1999b;  McPherson  &  Thomas,  1989;  Nielsen  & 
McPherson,  2001]).  In  general,  the  conceptual  framework  behind  marksmanship 
research  has  been  driven  by  improving  training  on  the  "fundamentals  of  rifle 
marksmanship"— the  physical  and  mental  factors  believed  to  underlie  skilled 
shooting  performance.  The  basic  set  of  relationships  among  postural  stability,  rifle 
handling,  distance,  and  weather  were  established  by  World  War  I.  For  example,  the 
1916  Marine  Corps  Score  Book  (Harllee,  1916)  describes  the  set  of  marksmanship 
fundamentals  that  are  also  covered  in  the  current  Marine  Corps  Rifle  Marksmanship 
Manual  (U.S.  Marine  Corps,  2001b).  The  basic  procedures  for  aligning  sights, 
maximizing  position  stability,  and  establishing  and  maintaining  breath  and  trigger 
control  remain  essentially  unchanged.  Most  of  the  research  on  rifle  marksmanship 
has  been  conducted  by  the  U.S.  Army  where  the  focus  has  traditionally  been  on 
developing  and  evaluating  different  training  programs  (e.g.,  Evans,  Dyer,  & 
Hagman,  2000;  Evans  &  Osborne,  1998;  Evans  &  Schendel,  1984;  Hagman,  1998, 
2000;  Hagman,  Moore,  Eisley,  &c  Viner,  1987;  Hagman  &  Smith,  1999;  McGuigan, 
1953). 

Studies  that  have  explicitly  attempted  to  predict  rifle  performance  have  been 
conducted  in  the  context  of  evaluating  how  well  performance  on  a  rifle  simulator 
relates  to  actual  record-fire  performance  on  the  firing  range  (e.g.,  Hagman,  1998; 
Marcus  &  Hughes,  1979;  Schendel,  Heller,  Finley,  &  Hawley,  1985;  Smith  & 
Hagman,  2000;  Torre  et  al.,  1987).  Another  body  of  research  has  used  shooting  as  a 
platform  to  study  skilled  behavior  in  relation  to  psychophysiological  constructs  and 
measurements  (e.g..  Bird,  1987;  Hatfield,  Landers,  &  Ray,  1987;  Janelle  et  al.,  2000; 
Kerick,  Iso-Ahola,  &  Hatfield,  2000;  Konttinen  &  Lyytinen,  1992,  1993;  Konttinen, 
Lyytinen,  &  Konttinen,  1995). 

A  complementary  approach  that  we  have  adopted  is  to  conceptualize  the 
domain  of  rifle  marksmanship  performance  as  a  function  of  skill  and  environment. 
We  have  conceptualized  the  skill  component  as  being  composed  of  three  interrelated 
dimensions:  perceptual-motor,  cognitive,  and  affective.  The  environment 
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component  is  composed  of  two  factors  that  are  uncontrollable,  equipment  and 
environment. 

Unfortunately,  shooting  has  not  been  conceptualized  as  a  skilled  sport  and 
there  has  been  little  in  the  way  of  theoretical  development  to  point  to  the  set  of 
important  variables  or  processes  underlying  skilled  shooting  performance.  By 
extension,  prior  research  provides  little  specific  guidance  on  which  variables  to 
measure.  However,  one  conclusion  that  is  clear  from  prior  research  is  that 
consistently  hitting  the  same  area  of  a  target  is  difficult.  This  difficulty  is  presumably 
due  to  the  requirement  to  simultaneously  coordinate  gross-motor  control  of  body 
positioning  with  fine-motor  control  of  the  trigger  finger,  minute  movements  of  the 
hands,  elbows,  legs,  feet,  and  cheek,  and  perceptual  cues  related  to  the  target,  front 
and  rear  sights,  rifle  movement,  and  body  movement,  while  under  stressful 
conditions. 

Thus,  from  a  theoretical  stance,  little  is  known  about  which  variables 
differentiate  shooters  of  different  skill  levels;  how  perceptual-motor,  cognitive,  and 
affective  variables  relate  to  each  other  over  time;  how  fast  shooting  skill  decays  over 
time;  how  sensitive  shooting  performance  is  to  shooting  conditions;  or  the  relative 
importance  of  different  shooting  position  variables. 

Rifle  Marksmanship  as  a  Complex  Skill 

To  better  understand  rifle  marksmanship  performance,  we  have  adopted  the 
stages-of-processing  framework  of  skilled  learning  (Ackerman,  1987,  1992;  Fitts  & 
Posner,  1967;  Wrisberg,  2003).  The  general  notion  is  that  skilled  performance  evolves 
over  three  stages  of  development:  an  early  cognitive  phase,  an  intermediate 
associative  phase,  and  a  final  autonomous  phase  (Fitts  &  Posner,  1967).  During  the 
cognitive  stage,  trainees  are  in  the  process  of  learning  what  the  task  entails— the 
procedures,  relevant  perceptual  cues,  different  shooting  positions,  how  to 
coordinate  breathing  and  squeezing  the  trigger,  and  how  to  use  results  (i.e.,  where 
the  round  hit)  as  feedback.  Thus,  there  is  high  cognitive  demand  for  performers  to 
attend  to  multiple  stimuli  and  procedures.  During  the  cognitive  stage,  trainees 
typically  learn  about  the  task  through  individual  or  whole-group  instruction  or 
other  training  vehicles  (e.g.,  CD-ROM  instruction,  rifle  simulators).  Training 
involves  weapon  handling  and  the  fundamentals  of  marksmanship— what  aiming, 
breath  control,  trigger  control,  and  the  different  positions  are,  and  how  they  affect 
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shot  placement.  During  live-fire  practice,  trainees  commit  much  of  their  cognitive 
resources  on  learning  and  understanding  the  task  with  respect  to  performance. 

Some  examples  of  the  kinds  of  learning  activities  during  the  cognitive  phase  for 
rifle  marksmanship  would  be  learning  the  terminology,  learning  how  to  handle  the 
weapon,  and  learning  the  fundamentals  of  rifle  marksmanship.  Performance  during 
the  cognitive  phase  is  low,  error-prone,  and  inconsistent,  and  requires  conscious 
thought.  The  high  cognitive-processing  demands  imposed  by  the  task  also  make 
performance  sensitive  to  distractions  and  other  ongoing  activities.  Novice  shooters 
would  be  expected  to  have  a  poor  grasp  of  the  fundamentals,  score  low,  exhibit  poor 
coordination  and  integration  of  the  different  elements  of  the  fundamentals,  and  not 
be  able  to  recognize  correct  from  incorrect  positions.  Novice  shooters  would  also  be 
expected  to  be  more  sensitive  to  changes  in  the  environment  (e.g.,  weather, 
equipment  malfunction,  anxiety)  than  more  advanced  shooters. 

The  intermediate  stage  is  characterized  by  performers  knowing  what  is 
expected  of  the  task.  During  this  stage,  the  attentional  demands  of  the  task  are 
reduced  and  thus  trainees  can  focus  on  refining  the  motor  responses  to  the  task  and 
develop  and  test  techniques  to  improve  performance.  Practice  on  the  task  becomes 
more  refined  and  consistent  and  the  gross  errors  of  the  cognitive  phase  diminish. 
Speed  and  accuracy  on  the  task  improve  over  the  practice  period  as  coordination 
between  cognitive  and  motor  responses  improves.  During  this  stage  and  the  more 
advanced  automaticity  stage,  knowledge  becomes  increasingly  compiled  and  broad 
ability  measures  and  content-specific  abilities  become  less  influential  on 
performance  for  closed-ended  skills  such  as  marksmanship. 

Characteristics  of  the  associative  phase  for  rifle  marksmanship  would  include  a 
basic  knowledge  of  weapons  handling,  terminology,  and  the  fundamentals  of  rifle 
marksmanship.  Performance  during  the  associative  phase  would  show  rapid 
improvement  over  the  course  of  fire. 

In  the  final  stage,  the  performer  executes  the  skill  automatically.  Performance  is 
consistent  and  seemingly  effortless.  The  cognitive  load  on  performers  with  respect  to 
executing  the  task  is  lowered  (compared  to  other  stages),  thus  freeing  up  resources. 
Shooters  who  have  reached  the  autonomous  phase  could  be  expected  to  be  true 
experts  —  snipers  or  members  of  the  rifle  team,  for  example.  Performance  is 
consistent  and  robust  against  distractions.  There  may  be  increases  in  performance 
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but  the  rate  of  improvement  slows  over  time.  Very  few  individuals  are  expected  to 
reach  this  stage  without  deliberate,  effortful,  and  consistent  practice. 

Thus,  the  stages-of-processing  conception  provides  an  interpretive  framework 
to  understand  (a)  how  overall  rifle  marksmanship  performance  evolves  over  time, 
and  (b)  that  different  classes  of  variables  differentially  relate  to  performance  at  each 
stage  of  development. 


Understanding  Rifle  Marksmanship  Within  the  Stages-of-Processing  Framework 

In  this  section  we  first  review  the  past  research  on  predicting  shooting 
performance  and  interpret  the  findings  within  this  framework.  Figure  3  shows  the 
broad  categories  and  the  variables  from  prior  research  that  have  been  examined 
within  each  category.  This  broad  framework  was  used  to  guide  our  development  of 
assessments  of  rifle  marksmanship,  conditioned  by  feasibility  and  delivery 
constraints.  Table  1  and  Table  2  summarize  the  findings  of  studies  that  have 
explicitly  examined  predictors  of  record-fire  performance. 


Figure  3.  Overview  of  rifle  marksmanship  variables  examined  in  marksmanship  research. 
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Table  1 


Summary  of  Perceptual-Motor  Variables  Related  to  Record-Fire  Performance  (p  <  .05) 


Predictor  variables 

r 

n 

Rifle  steadiness  (Humphreys,  Buxton,  &  Taylor,  1936) 

.72 

43 

Rifle  steadiness  (McGuigan  &  MacCaslin,  1955) 

.22 

148 

Rifle  steadiness  (Spaeth  &  Dunham,  1921) 

.61 

73 

Prior  shooting  experience  and  aptitude  (MacCaslin  &  McGuigan,  1956) 

R  =  .67  to  .72 

Prior  shooting  experience  (Tierney,  Cartner,  &  Thompson,  1979),  males 

.24 

Prior  shooting  experience  (Tierney  et  al.,  1979),  females 

.19 

Prior  hunting  experience  (Tierney  et  al.,  1979),  males 

.21 

Prior  experience  with  a  .22  rifle  (Thompson,  Smith,  Morey,  &  Osborne, 

1980),  males 

.21  to  .25 

Prior  record-fire  performance  (McGuigan  &  MacCaslin,  1955).  Repeated 
qualification  course  3  times. 

.88  and  .84 

Self-reported  prior  record-fire  performance  (Schendel  et  al.,  1983) 

.29 

121 

Prior  record-fire  performance  (Smith,  2000,  Experiment  2) 

.37 

50 

Prior  record-fire  performance  (Thompson,  Morey,  Smith,  &  Osborne,  1981) 

388 

Weaponeer  device-fire  (Schendel  et  al.,  1985,  Experiment  1) 

.37-.74 

102 

Weaponeer  device-fire  (Schendel  et  al.,  1985,  Experiment  2) 

.17-.55 

244 

Combat  Training  Theater  device-fire  (Marcus  &  Hughes,  1979) 

low 

Laser  Marksmanship  Training  System  device-fire  (Smith  &  Hagman,  2000) 

.55 

95 

Engagement  Skills  Trainer  device-fire  (Hagman,  1998) 

.68 

102 

Other  device-fire  (Torre  et  al,  1987) 

.54 

29 
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Table  2 

Summary  of  Cognitive  and  Affective  Variables  Related  to  Record-Fire  Performance  (p  <  .05  unless 
otherwise  noted) 

Predictor  variables 

r 

n 

Cognitive 

ASVAB  clerical/ administrative  (Carey,  1990) 

.26§ 

ASVAB  general  technician  (Carey,  1990) 

.35§ 

ASVAB  electrical  repair  (Carey,  1990) 

.32§ 

ASVAB  mechanical  maintenance  (Carey,  1990) 

.38§ 

ASVAB  composite  (Carey,  1990) 

.32§ 

Infantry  training  GPA  (Carey,  1990) 

.25§ 

Core  job  knowledge  (Carey,  1990) 

.31§ 

Supervisor  ratings  (Carey,  1990) 

.16§ 

Skill  Qualification  Test,  a  measure  of  a  soldier's  skill  achievement  (Wisher, 
Sabol,  Sukenik,  &  Kern,  1991) 

.24 

439 

Armed  Forces  Qualification  Test  (Wisher  et  al.,  1991) 

n.s. 

Knowledge  of  zeroing  (Thompson  et  al.,  1980) 

.50 

144 

Knowledge  of  bullet  strikes  at  greater  ranges  than  the  25m  target  trainees 
were  practicing  on  (Thompson  et  al.,  1980) 

.33 

144 

Knowledge  of  distance  effects  and  appropriate  sight  adjustments  (Thompson 
et  al.,  1980) 

.31 

144 

Anxiety 

Self-reported  nervousness  (Tierney  et  al.,  1979) 

-.19 

Predicted  record-fire  score  for  soldiers  whose  confidence  estimates  in  their 
prediction  were  >  90%  (Schendel  et  al.,  1983) 

.50 

41 

^Significance  level  not  reported. 


Perceptual-motor  variables.  The  perceptual-motor  variables  under 
consideration  relate  to  the  physical  aspects  of  shooting  such  as  carrying  out  the 
different  shooting  positions,  establishing  proper  sight  alignment  and  sight  picture, 
and  maintaining  rifle  steadiness.  Skilled  shooters  are  able  to  position  various  body 
parts  to  achieve  maximum  rifle  support  with  minimal  fatigue  in  different  positions 
(prone,  sitting,  kneeling,  standing),  and  establish  and  maintain  sight  alignment 
(centering  the  front  sight  post  within  the  rear  sight  aperture)  and  correct  sight 
picture  (centering  the  sights  on  the  target).  A  shooter's  skill  in  consistently  hitting 
the  same  spot  on  a  target  is  determined  largely  by  the  extent  to  which  he  or  she  can 
maintain  these  factors  before,  during,  and  after  firing  a  round. 
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Steadiness.  For  example,  skilled  shooters  have  been  found  to  be  able  to  hold  a 
rifle  steadier  than  unskilled  shooters  and  this  steadiness  relates  positively  to 
shooting  performance  (Humphreys  et  al.,  1936;  McGuigan  &  MacCaslin,  1955; 
Spaeth  &  Dunham,  1921).  In  general,  being  able  to  maintain  a  steady  body  position 
has  consistently  been  found  to  be  related  to  shooting  performance.  Expert  shooters 
have  been  found  to  perform  higher  on  measures  of  whole-body  stability  (e.g..  Era, 
Konttinen,  Mehto,  Saarela,  &  Lyytinen,  1996;  Gates,  1918).  While  an  obvious  finding, 
what  is  less  obvious  is  that  experts  have  been  found  to  increase  their  stability  during 
the  aiming  period  preceding  the  shot  (Era  et  al.,  1996). 

Experience  with  weapons.  Another  important  variable  that  has  been  found 
related  to  shooting  performance  is  experience  with  weapons  and  prior  record-fire 
performance.  MacCaslin  and  McGuigan  (1956)  found  self-reported  shooting 
experience  combined  with  aptitude  scores  contributed  substantially  to  record-fire 
score  prediction,  with  multiple  R  between  .67  and  .72,  p  <  .01.  Similarly,  Tierney  et 
al.  (1979)  found  low,  positive  relationships  between  self-reported  prior  rifle 
experience  and  record-fire  scores  for  males  and  females  (r  =  .24,  p  <  .05;  r  =  .19,  p  < 
.05).  In  addition,  hunting  experience  also  correlated  significantly  with  record-fire 
scores  for  males  (r  =  .21,  p  <  .05).  Similarly,  significant  relations  were  found  between 
self-reported  experience  with  a  .22  caliber  rifle  and  record-fire  scores  for  male  entry- 
level  Army  trainees  (r  ranged  from  .21  to  .25,  depending  on  the  experimental 
condition,  Thompson  et  al.,  1980). 

Previous  record-fire  performance.  In  addition  to  experience  with  weapons,  one  of 
the  best  predictors  of  a  record-fire  score  is  the  shooter's  previous  record-fire  score. 
McGuigan  and  MacCaslin  (1955)  reported  test-retest  reliabilities  of  unskilled  Army 
trainees  on  three  trials  of  a  slow-fire  qualification  course.  The  average  reliability  of 
the  slow-fire  course  for  samples  drawn  from  two  Army  bases  was  .88  and  .84. 
Schendel  et  al.  (1983)  report  a  significant  correlation  between  Army  soldiers'  self- 
reported  prior  record-fire  score  and  actual  record-fire  score  (r  =  .29,  n  =  121,  p  <  .01). 
Smith  (2000,  Experiment  2)  found  a  correlation  of  .37  (p  <  .05,  N  =  50)  between  the 
prior  year's  record-fire  score  and  the  current  year's  record-fire  score  for  U.S.  Army 
Reserve  soldiers.  As  part  of  a  study  of  skill  retention,  soldiers  were  retested  6  weeks 
after  completing  the  basic  rifle  marksmanship  course  (Thompson  et  al.,  1981). 
Soldiers  (N  =  388)  repeated  the  same  qualification  course  six  weeks  after  record 
qualification.  No  correlations  between  record-fire  and  retest  scores  were  reported; 
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however,  compared  to  the  record-fire  score,  33%  of  soldiers  scored  lower  and  60%  of 
soldiers  scored  higher. 

Device-fire  performance.  Device-fire  performance,  or  shooting  performance  on  a 
rifle  simulator,  has  been  one  of  the  strongest  predictors  of  record-fire  performance. 
The  use  of  rifle  simulators  has  received  much  attention  because  of  the  cost-savings 
potential  for  sustainment-level  training  and  remediation. 

One  of  the  earliest  rifle  simulator  systems  was  Weaponeer.  Schendel  et  al. 
(1985,  Experiment  1)  examined  the  relationship  between  device-fire  scores  on 
Weaponeer  and  record-fire  scores  1  to  2  days  later  (foxhole  supported  position  only). 
Schendel  et  al.  (1985)  found  correlations  between  .37  and  .74,  depending  on  the 
experimental  condition.  However,  in  a  follow-up  study,  the  authors  found  lower 
correlations,  between  .17  and  .55.  Similar  results  were  found  for  the  Laser 
Marksmanship  Training  System  (LMTS).  Smith  and  Hagman  (2000)  found  a 
moderate  correlation  between  LMTS  device-fire  scores  and  range  record-fire  scores 
(r  =  .55,  p  <  .05,  N  =  95).  Likewise,  Hagman  (1998)  found  a  strong  correlation 
between  record-fire  scores  and  device-fire  scores  using  the  Engagement  Skills 
Trainer  (r  =  .68,  p  <  .05,  N  =  102).  Finally,  a  moderate  correlation  was  found  by  Torre 
et  al.  (1987)  in  their  investigation  (r  =  .54,  p  <  .05,  n  =  29).  Other  systems  have  been 
investigated  (e.g..  Combat  Training  Theater,  Marcus  &  Hughes,  1979),  but  low 
correlations  between  device-fire  and  record-fire  scores  have  been  found,  possibly 
owing  to  poor  live-fire  range  conditions. 

Summary.  Some  of  the  strongest  predictors  of  shooting  performance  are 
experience-related  variables.  Surprisingly,  the  magnitude  of  the  relationship  is  lower 
than  might  be  expected  and  the  correlation  between  the  experience-related  variables 
and  shooting  score  varies  considerably  across  studies.  For  example,  the  test-retest 
reliability  of  previous  shooting  experience  is  at  best  in  the  .80s  (McGuigan  & 
MacCaslin,  1955)  across  the  span  of  a  few  days.  As  the  time  interval  increases  (e.g., 
across  annual  qualifications),  the  correlation  drops  to  the  .30s.  Just  as  surprising  are 
the  range  of  correlations  between  device-fire  and  record-fire  scores  (r  ranges  from  .2 
to  .7)  over  one  or  two  days.  Our  interpretation  of  these  results  is  that  in  general, 
shooting  performance  is  very  sensitive  to  many  variables.  While  perceptual-motor 
variables  may  be  the  strongest  predictor  of  shooting  performance,  cognitive  and 
affective  variables  may  help  explain  the  variation. 
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Cognitive  variables.  Presumably  underlying  shooting  performance  is  the 
knowledge  of  marksmanship  —  (a)  knowing  position-related  factors  (e.g.,  what  the 
different  positions  are,  how  to  properly  position  limbs,  how  to  control  breathing  and 
trigger,  how  to  align  the  sights  on  the  target),  and  (b)  knowing  cause-effect  relations 
(e.g.,  knowing  how  yanking  the  trigger  affects  the  muzzle,  knowing  why  the  rifle 
should  be  supported  by  bone  instead  of  muscle,  understanding  the  concept  that 
everything  a  shooter  does  position-wise  should  minimize  body  movement  [and  thus 
rifle  movement]). 

The  importance  of  knowledge  of  marksmanship  was  recognized  by  early 
authors  on  shooting.  For  example,  Whelen  (1918,  p.  455)  asserted  that  "Rifle 
shooting  is  almost  entirely  a  matter  of  intelligent  practice.  Practice  alone,  without 
head  work,  will  not  get  one  very  far."  Similarly,  the  1916  Marine  Corps  Score  Book 
emphatically  stated  that  Marines  should  not  even  be  allowed  to  handle  a  weapon  if 
they  did  not  know  the  fundamentals  of  marksmanship  (Harllee,  1916).  However, 
rifle  marksmanship  research  has  virtually  ignored  the  simple  questions  of  how 
much  do  shooters  know  about  marksmanship  and  how  does  their  knowledge  of 
marksmanship  relate  to  their  shooting  performance? 

Training  ejfects.  The  available  evidence  suggests  that  shooting  performance  is 
sensitive  to  knowledge.  Studies  of  different  training  programs  clearly  show  group 
differences  in  record-fire  scores.  For  example,  McGuigan  (1953)  compared  part-  and 
whole-task  training  methods  with  Army  basic  trainees.  Those  soldiers  in  the  whole- 
task  training  condition  outperformed  soldiers  in  the  part-task  training  condition  on 
record-fire  performance.  Further,  regardless  of  training  condition,  soldiers  in  general 
shot  higher  after  receiving  training. 

Additional  evidence  of  the  sensitivity  of  shooting  to  instruction  is  seen  in  an 
instructional  study  by  Boyce  (1987).  In  her  study,  Boyce  compared  the  performance 
of  unskilled  (novice)  shooters  in  the  prone  position  across  three  instructional 
conditions  and  a  control  condition  over  five  days  (trials).  The  experimental 
conditions  received  different  forms  of  instruction  on  marksmanship.  The  control 
condition  received  the  minimal  instruction  required  to  safely  shoot  the  weapon.  In 
general,  conditions  with  instruction  on  marksmanship  significantly  outperformed 
the  control  group.  In  addition,  a  main  effect  for  trial  was  found,  with  the  mean  score 
across  conditions  on  the  first  trial  significantly  lower  than  the  mean  score  on 
subsequent  trials.  Similarly,  higher  rifle  marksmanship  performance  was  observed 
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in  normal  versus  accelerated  training  cycles,  again  suggesting  sensitivity  to 
instruction  and  thus  a  cognitive  component  (Cline,  Beals,  &  Seidman,  1960). 

Knowledge  of  shooting.  In  a  study  comparing  different  marksmanship  training 
techniques,  Thompson  et  al.  (1980)  found  that  for  entry-level  male  Army  trainees, 
record-fire  scores  correlated  with  knowledge  of  zeroing  (r  =  .50,  p  <  .001,  n  =  144), 
knowledge  of  bullet  strikes  at  greater  ranges  than  the  25m  target  trainees  were 
trained  on  (r  =  .33,  p  <  .001,  n  =  144),  and  knowledge  of  range  effects  and 
appropriate  sight  adjustments  (r  =  .31,  p  <  .001,  n  =  144).  However,  this  finding  was 
only  for  the  experimental  treatment  receiving  the  most  training  support.  A  similar 
pattern  of  correlations  but  lower  in  magnitude  (r  in  the  low  .20s)  was  found  for 
another  experimental  condition.  However,  no  such  relations  were  found  for  the 
normal  training  condition  or  for  a  third  training  condition. 

Aptitude  and  achievement.  Other  studies  have  examined  the  relationship  between 
aptitude  and  achievement  measures  with  record-fire  performance.  While  not 
measures  of  marksmanship  knowledge,  these  variables  suggest  the  potential  for 
shooters  to  acquire  the  knowledge. 

Carey  (1990)  examined  the  relationship  between  a  variety  of  background 
variables  and  first-term  enlisted  Marines'  known  distance  (KD)  record-fire 
performance.  Moderate  correlations  between  various  subscales  of  the  ASVAB  and 
record-fire  scores  were  found,  ranging  from  .26  for  the  clerical/ administrative 
subscale  to  .38  on  the  mechanical  maintenance.  The  relationship  to  the  overall 
ASVAB  composite  was  .32.  Infantry  training  GPA  was  weakly  related  to  record-fire 
scores  (r  =  .25),  as  was  performance  on  a  core  job  knowledge  test  (r  =  .31)  and 
supervisor  ratings  (r  =  .16).  Unfortunately,  Carey  does  not  report  significance  levels 
of  these  correlations. 

Similarly,  a  significant  correlation  was  found  between  record-fire  score  and 
score  (r  =  .24,  p  <  .01,  n  =  439)  on  the  Skill  Qualification  Test,  a  measure  of  a  soldier's 
skill  achievement  (Wisher  et  al.,  1991).  However,  Wisher  et  al.  found  no  relationship 
between  soldiers'  scores  on  the  Armed  Forces  Qualification  Test  (AFQT),  a  measure 
of  aptitude,  and  their  record-fire  scores.  As  mentioned  earlier,  MacCaslin  and 
McGuigan  (1956)  combined  aptitude  (Aptitude  Area  I  on  the  Army  Classification 
Battery)  and  self-reported  shooting  experience  to  predict  record-fire  score  with  a 
multiple  R  in  the  .70  range. 
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Summary.  While  knowledge  of  rifle  marksmanship  has  been  recognized  as 
important  for  nearly  a  century,  there  has  been  virtually  no  research  examining  the 
relationship  between  knowledge  of  rifle  marksmanship  and  shooting  performance. 
The  only  study  that  did  examine  what  was  learned  from  training  found  moderate 
relationships  between  knowledge  and  shooting  performance.  Training  clearly  has  an 
impact  on  shooting  performance,  and  proxy  variables  for  the  potential  to  learn  such 
knowledge  are  suggestive  of  a  relationship.  These  findings  point  to  a  possible 
knowledge  component  to  shooting  performance. 

Affective  variables.  As  with  knowledge  variables,  the  relationship  between 
affective  variables  and  shooting  performance  has  been  largely  unexplored.  In 
general,  the  amount  and  type  of  mental  thoughts  preceding  the  moment  of  firing  are 
believed  to  have  an  influence  on  shot  quality.  For  example,  EEG  frequencies 
preceding  low-scoring  shots  in  an  expert  shooter  were  interpreted  as  resulting  from 
distracting  thoughts  and  increased  mental  activity  (Bird,  1987;  Konttinen  & 
Lyytinen,  1992).  This  interpretation  is  consistent  with  an  early  examination  of  expert 
and  novice  shooters  (Gates,  1918),  where  novice  shooters'  performance  was  affected 
severely  by  dwelling  on  steadiness  factors  (e.g.,  "I  can't  seem  to  control  myself"  or 
"There,  I  moved  again";  p.  3).  Tierney  et  al.  (1979)  found  low,  negative  relationships 
between  self-reported  nervousness  about  firing  and  record-fire  scores  for  females 
but  not  males  (r  =  -.19,  p  <  .05).  Sade,  Bar-Eli,  Bresler,  &  Tenenbaum  (1990)  found 
that  highly  skilled  shooters  reported  significantly  lower  (state)  anxiety  than 
moderately  skilled  shooters  when  measured  10  minutes  prior  to  competition  (seven 
occasions).  Further,  shooting  performance  was  negatively  related  with  state  anxiety 
in  six  of  seven  competitions. 

Equipment  and  environmental  variables.  Finally,  a  third  area  that  has 
received  attention  is  in  investigating  equipment  aspects.  For  example,  Kemnitz,  Rice, 
Irwin,  Merullo,  and  Johnson  (1997)  found  that  shooting  performance  on  an  M16A2 
(as  measured  using  a  rifle  simulator)  increased  with  a  shorter  stock  and  reduced 
rifle  weight.  Early  studies  of  the  M16A1  rifle  examined  performance  variables  such 
as  accuracy  of  the  rifle,  zeroing,  shooter  error,  barrel  stress,  and  ballistics.  Osborne, 
Morey,  and  Smith  (1980)  and  later  studies  compared  firing  and  serviceability 
characteristics  of  the  M16A1  to  M16A2  (Osborne  &  Smith,  1986). 

Summary  and  discussion.  In  general,  the  highest  correlations  with  record-fire 
scores  were  with  steadiness  scores  and  live-fire  and  device-fire  scores.  Prior  record- 
fire  scores  correlated  lower  with  current  record-fire  scores,  possibly  owing  to  skill 
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decay  over  time  periods  of  12  months.  Prior  shooting  experience  related  with 
record-fire  scores,  but  the  correlations  were  low  in  magnitude.  Knowledge-related 
variables  correlated  moderately  with  record-fire  scores,  and  prior  shooting 
experience  scores  correlated  even  lower. 

When  the  data  are  interpreted  within  the  stages-of-skill-processing  framework, 
several  trends  appear  that  are  consistent  with  the  framework.  First,  the  overall 
variability  in  shooting  performance  across  studies  can  be  interpreted  as  sensitivity  of 
the  act  of  shooting  to  different  conditions.  The  strongest  evidence  that  shooting  can 
be  consistent  is  in  the  test-retest  study  by  McGuigan  and  MacCaslin  (1955).  In  their 
studies,  the  shooters  fired  the  same  qualification  course  three  times  across  three 
days.  The  test-retest  coefficient  was  .84  and  .88  depending  on  the  sample. 

One  of  the  most  interesting  findings  is  the  low  predictability  of  the  most  recent 
record-fire  score  (r  in  the  .3-. 4  range).  Again,  when  interpreted  in  terms  of  the 
stages-of-skill-processing  framework,  the  data  are  consistent  with  the  idea  that 
shooters  were  still  in  the  cognitive  phase.  The  sample  was  young  soldiers  or  Marines 
who  were  undergoing  sustainment-level  training  and  requalification.  The  typical 
time  in  service  was  1-2  years.  Performance  was  varied  and  inconsistent. 

The  low  to  moderate  correlations  between  device-fire  and  record-fire  scores 
observed  across  different  rifle  simulators  may  reflect  a  possible  effect  due  to  fidelity 
of  the  shooting  context.  That  is,  the  simulator  system  only  approximates  shooting 
conditions.  The  simulators  in  the  studies  reviewed  in  this  report  are  all  intended  to 
be  used  indoors.  Indoor  conditions  remain  stable  compared  to  the  outdoors,  which 
can  vary  considerably  over  time  (e.g.,  cloud  cover,  intensity  of  sunlight  [time  of 
day],  temperature,  and  humidity).  The  act  of  firing  differs  as  well.  Recoil  and  report 
are  not  simulated  except  for  the  Engagement  Skills  Trainer  (EST)  system,  which  also 
found  the  highest  correlations  with  record-fire  scores.  Finally,  the  consequences  of 
poor  performance  were  low  for  device-fire  and  relatively  high  for  record-fire.  The 
studies  were  conducted  with  volunteers  and  device-fire  scores  were  not  part  of  the 
participant's  permanent  record.  One  effect  of  this  difference  was  that  the  shooters 
probably  did  not  experience  as  much  "match  pressure"  during  device-fire  as  they 
did  during  record-fire. 

Consistent  with  the  idea  of  more  varied  performance  in  the  early  stages  of  skill 
learning  compared  to  later  stages,  higher  correlations  between  device-fire  and 
record-fire  were  found  for  the  presumably  more  experienced  sample.  The  sample  in 
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Smith  and  Hagman  (2000)  were  U.S.  Army  Reservists  whereas  other  studies  used 
entry-level  trainees.  In  this  case,  the  entry-level  trainees  would  be  expected  to  show 
inconsistent  performance  on  both  the  rifle  simulator  and  for  record-fire,  which  may 
explain  the  lower  correlations. 

Overall,  the  research  reviewed  on  predicting  shooting  performance  suggests  a 
deceptively  complex  task  sensitive  to  a  variety  of  variables.  There  appears  to  be  a 
strong  perceptual-motor  component  that  includes  motor  control  and  experience 
variables.  Further,  shooting  performance  also  appears  to  be  sensitive  to  affective 
variables  via  the  influence  of  anxiety  on  motor  control.  There  also  appears  to  be  an 
aptitude  and  knowledge  component  associated  with  shooting  performance; 
however,  this  area  appears  to  be  largely  unexplored  beyond  examination  of 
shooting  performance  across  groups  receiving  different  training  and  instruction. 

Validation  Strategy 

Prior  rifle  marksmanship  research  has  examined  tangentially  the  role  of 
knowledge  in  shooting  performance.  However,  while  there  exists  no  research 
examining  the  relationships  among  perceptual-motor,  cognitive,  and  affective 
variables  for  Marines  in  the  learning  phase  and  past  the  learning  phase,  research  in 
other  sports  suggests  the  existence  of  a  relationship  between  cognitive  and  motor 
skills  (McPherson  &  French,  1991;  Thomas,  French,  &  Humphries,  1986).  Thus,  much 
of  the  work  reported  herein  is  an  attempt  to  gather  validity  evidence  to  address 
these  issues. 

Our  validation  approach  was  to  gather  a  range  of  evidence  that  would  provide 
information  about  the  suitability  of  our  assessments  for  the  purpose  of  identifying 
Marines  who  would  be  at  risk  of  failing  qualification.  However,  because  very  few 
Marines  fail  qualification  (and  the  sample  used  in  our  data  collection  reflected  this), 
we  instead  addressed  the  broader  question  of  predicting  record-fire  score.  In 
addition,  given  that  the  measures  we  developed  were  novel  and  their  application  to 
rifle  marksmanship  the  first  we  knew  of,  we  gathered  a  wide  range  of  validity 
evidence  that  would  provide  information  about  the  quality  of  our  measures.  Each 
type  of  evidence  is  described  below. 

Evidence  of  knowledge  and  skill  performance  consistent  with  the  skill 
acquisition  model.  An  important  piece  of  evidence  of  the  suitability  of  our 
assessment  approach  would  be  in  findings  that  are  consistent  with  the  skill- 
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acquisition  model.  In  particular,  perceptual-motor  variables  should  matter  more  for 
prediction  for  more  experienced  participants  (vs.  less  experienced),  and  aptitude 
and  domain-related  variables  should  matter  more  for  less  experienced  participants 
(vs.  more  experienced). 

Evidence  of  the  predictability  of  record-fire  scores.  Another  important  piece 
of  evidence  would  be  in  achieving  high  predictability  of  record-fire  scores  using  our 
set  of  measures  as  dependent  variables.  If  the  set  of  perceptual-motor,  cognitive,  and 
affective  measures  — all  ADL  compatible— could  predict  Marines'  record-fire  scores 
reasonably  well,  then  that  finding  would  be  compelling  evidence  for  our  approach. 

Evidence  of  a  relationship  among  knowledge  measures.  In  addition  to 
construct  and  predictive  validity  of  our  measures,  there  should  also  be  evidence  that 
our  knowledge  measures  relate  to  each  other;  however,  because  the  measures  target 
different  areas  of  marksmanship  knowledge,  we  expected  the  measures  to  be 
differentially  sensitive  to  Marines  with  different  levels  of  experience. 

Evidence  of  the  sensitivity  of  knowledge  measures.  Another  important  body 
of  evidence  that  would  provide  information  on  the  quality  of  our  measures  is  the 
sensitivity  of  our  measures  to  (a)  instructional  effects  and  (b)  knowledge  differences. 
With  respect  to  instructional  sensitivity,  one  criterion  of  a  sensitive  instrument  is  its 
capability  to  detect  changes  in  knowledge  when  learning  occurs;  that  is,  the  measure 
should  yield  lower  scores  on  a  pretest  and  higher  scores  on  a  posttest,  where  the 
intervening  event  is  instruction.  Related  to  this  is  the  idea  that  the  measure  should 
yield  higher  scores  for  those  who  have  more  knowledge  than  for  those  with  less 
knowledge. 


Research  Questions 

As  suggested  by  the  review  on  rifle  marksmanship  research,  a  variety  of 
variables  have  been  found  to  relate  to  shooting  performance,  including  perceptual- 
motor,  cognitive,  and  affective  variables.  The  skill-acquisition  model  suggests 
poorest  performance  during  the  learning  phase  (i.e.,  when  trainees  are  least  likely  to 
have  acquired  and  internalized  the  knowledge  required  to  shoot  well).  This 
framework  guided  our  measurement  strategy  in  the  following  way.  First,  we 
assumed  that  shooting  skill,  as  measured  by  record-fire  scores,  followed  the  skill- 
acquisition  model;  thus,  we  expected  that  consistently  lower  shooting  scores  would 
be  an  indication  that  Marines  were  in  the  learning  phase  and  consistently  higher 
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shooting  scores  would  be  an  indication  Marines  were  past  the  learning  phase. 
Second,  because  we  assumed  a  relationship  between  low  performance  and  learning, 
we  focused  on  developing  or  adopting  measures  that  would  differentiate  Marines  in 
the  learning  phase  from  Marines  past  the  learning  phase.  We  briefly  describe  next 
how  we  operationalized  each  type  of  variable  and  then  present  the  research 
questions. 

Perceptual-motor.  We  defined  the  perceptual-motor  construct  for  rifle 
marksmanship  as  the  combination  of  perceptual,  gross-motor,  and  fine-motor  skills 
related  to  successfully  shooting  a  rifle.  While  measures  of  steadiness  have  been  used 
in  prior  research  and  have  shown  a  relationship  with  record-fire  scores,  our 
assumption  was  that  such  measures  would  not  be  feasible  in  a  fielded  setting  and 
thus  were  ruled  out.  Instead,  we  developed  survey  questions  that  would  serve  as 
indirect  measures  of  perceptual-motor  development. 

We  expected  the  perceptual-motor  component  to  function  differently 
depending  on  where  the  Marine  was  in  the  skill-acquisition  phase.  For  the  current 
studies,  we  measured  the  perceptual-motor  construct  via  proxy  variables  such  as 
prior  record-fire  scores,  prior  shooting  experience,  shooting  frequency,  and 
competitive  shooting  experience. 

Cognitive.  We  operationalized  the  cognitive  construct  as  (a)  aptitude  and  (b) 
knowledge  of  rifle  marksmanship.  Given  the  ADL  environment,  we  targeted  the 
knowledge  component  for  measurement  as  knowledge  was  expected  to  be  the  most 
variable  for  Marines  in  the  learning  phase  of  the  skill-acquisition  model.  In  addition, 
we  expected  that  measuring  the  knowledge  component  would  be  a  tractable 
problem  in  an  ADL  context. 

We  used  the  CRESST  development  model  to  focus  on  the  cognitive  demands 
underlying  rifle  marksmanship.  Cognitive  demands  refers  to  the  set  of  skills, 
knowledge,  and  abilities  (KSA)  underlying  successful  rifle  marksmanship.  In  the 
current  study,  the  KSAs  were  the  targets  of  assessment,  particularly  knowledge. 
Within  that  context,  we  developed  assessments  that  we  expected  to  be  differentially 
sensitive  to  shooters  in  the  learning  phase  compared  to  shooters  past  the  learning 
phase.  That  is,  we  expected  that  higher  performing  shooters  would  have  a  deeper 
understanding  and  a  broader  knowledge  base  about  rifle  marksmanship  compared 
to  lower  performing  shooters. 
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Affective.  We  defined  the  affective  construct  for  rifle  marksmanship  as  the  set 
of  worry  and  anxiety  constructs  related  to  successful  rifle  marksmanship.  Prior 
research  suggests  a  relationship  between  affective  variables  and  record-fire  scores. 
Anecdotal  evidence  (i.e.,  discussion  with  marksmanship  coaches)  also  suggests  that 
anxiety  and  worry  are  negatively  related  to  shooting  performance,  possibly  owing  to 
their  effect  on  physiological  processes  (e.g.,  increased  heart  rate,  breathing)  and 
subsequently  on  maintaining  a  steady  rifle. 

Thus,  our  research  was  organized  around  questions  related  to  predicting 
record-fire  scores  and  questions  related  to  the  quality  of  our  measures.  The  first  two 
research  questions  bear  directly  on  rifle  marksmanship.  The  last  research  question 
focuses  on  the  quality  of  assessments. 

To  what  extent  can  record-fire  performance  be  predicted  from  knowledge, 
perceptual-motor,  and  affective  variables? 

•  Given  the  skill-acquisition  framework,  we  expected  that  in  general,  higher 
shooting  performance  would  be  associated  with  (a)  a  deeper  understanding 
and  a  broader  knowledge  of  rifle  marksmanship,  (b)  more  shooting 
experience,  and  (c)  consistent  shooting  performance. 

•  For  shooters  in  the  learning  phase,  we  expected  higher  shooting 
performance  to  be  associated  with  (a)  higher  aptitude,  (b)  more  knowledge 
about  rifle  marksmanship,  and  (c)  inconsistent  performance. 

•  Given  prior  research,  we  expected  negative  affect  to  be  negatively 
associated  with  shooting  performance. 

What  combination  of  perceptual-motor,  cognitive,  and  affective  measures 
yield  the  best  predictors  of  record-fire  performance? 

•  Given  prior  research,  we  expected  in  general  that  perceptual-motor 
variables  would  contribute  the  most  to  the  overall  prediction  of  shooting 
performance. 

•  Given  the  skill-acquisition  framework,  we  expected  (a)  the  perceptual- 
motor  component  to  contribute  more  to  the  prediction  of  shooting 
performance  for  Marines  past  the  learning  phase  than  for  Marines  in  the 
learning  phase;  and  (b)  the  knowledge  component  to  contribute  more  to  the 
prediction  of  shooting  performance  for  Marines  in  the  learning  phase  than 
for  Marines  past  the  learning  phase. 
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•  Given  the  skill-acquisition  framework,  which  does  not  address  affect,  we 
had  no  expectations  of  the  degree  to  which  affective  variables  would 
contribute  to  the  prediction  of  shooting  performance. 

What  is  the  quality  of  the  validity  evidence  for  our  measures  of  rifle 
marksmanship  knowledge? 

•  An  important  criterion  for  judging  the  quality  of  an  assessment  is  the  extent 
to  which  the  expectations  listed  in  the  prior  two  research  questions  are 
borne  out.  Because  these  expectations  are  derived  directly  from  a  theoretical 
model  and  prior  research,  results  consistent  with  a  theoretical  framework 
would  be  evidence  that  our  assessments  are  operating  as  expected. 

•  Other  criteria  for  judging  the  quality  of  our  assessments  are  (a)  the  extent  to 
which  the  knowledge  measures  relate  to  each  other;  and  (b)  the  degree  to 
which  the  assessments  are  able  to  detect  differences  due  to  instruction  and 
differences  due  to  presumed  differences  in  knowledge. 

SECTION  II:  MEASUREMENT  APPROACH  AND  OVERVIEW  OF  STUDIES 

Section  II  provides  our  general  approach  to  developing  measures,  a  brief  listing 
of  the  studies  conducted,  and  a  brief  summary  of  the  findings  of  the  different  studies 
with  respect  to  the  research  questions  posed  earlier. 

Our  general  measurement  approach  was  to  develop  a  range  of  measures  to 
assess  the  different  kinds  of  knowledge  expected  of  Marines  at  different  competency 
levels.  Before  we  could  develop  the  measures  however,  we  first  had  to  understand 
the  terminology,  concepts,  and  causal  relations  within  the  domain  of  rifle 
marksmanship. 


Domain  Sampling 

Our  analysis  of  the  domain  resulted  in  identifying  119  key  knowledge 
components  culled  from  the  USMC  rifle  marksmanship  doctrine  manual  (USMC, 
2001b).  The  knowledge  components  are  given  in  Appendix  A.  The  knowledge  was 
separated  into  37  factual  and  38  causal  knowledge  components.  Further  refinement 
of  the  knowledge  components  resulted  in  identifying  key  factual  knowledge 
(Appendix  B)  and  key  causal  knowledge.  Factual  knowledge  covered  basic 
definitions  and  procedures  and  in  general,  could  be  acquired  via  rote  memorization. 
Causal  knowledge  covered  cause-effect  relations  (Appendix  C)  and  was  considered 
more  conceptual.  To  determine  which  knowledge  components  to  sample  in  our 
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assessments,  we  asked  the  marksmanship  coaches  at  Stone  Bay  to  identify  the  topics 
most  relevant  to  shooting  performance.  As  a  result  of  this  consultation,  we 
eliminated  topics  related  to  weather  or  the  environment  (e.g.,  effects  of  overcast), 
uniform  (e.g.,  effects  on  placement  of  the  rifle  butt  in  the  shoulder),  ammunition, 
physiology  (e.g.,  perspiration,  rapid  fatigue),  and  equipment  (e.g.,  effects  of  rifle 
chamber  temperature).  The  remaining  knowledge  elements  were  the  major  elements 
presumed  to  be  most  directly  related  to  placing  a  round  on  target.  Table  3  shows  the 
topics  and  subtopics  that  were  sampled  across  the  assessments. 

Table  3 

Topics  Sampled  in  Assessments 

Topic  Subtopics 

Breath  control  Breath  control.  Natural  Respiratory  Pause,  Natural  point  of  aim 

Trigger  control  Bucking,  Finger  placement.  Firm  grip.  Flinching,  Grip  of  firing  hand.  Trigger  control 
Trigger  squeeze 

Aiming  Accuracy,  Aiming  process.  Follow-through,  Eye  on  front  sight  post.  Sight  adjustment. 

Sight  alignment/ picture 

Position  7  factors  common  to  all  shooting  positions.  Body  placement,  Bone  support,  Eye  relief. 

Feet  placement.  Finger  placement.  Firm  grip.  Forward  elbow  placement.  Forward 
hand  placement.  Leg  placement.  Muscular  relaxation.  Muscular  tension.  Rifle  butt 
placement.  Stable  firing  position.  Stock  weld  placement 

Other  Consistency,  Distance  effects.  Weapons  safety,  BZO  setting 

Given  the  domain,  our  general  measurement  strategy  was  to  develop  a  range 
of  measures  for  different  kinds  of  knowledge  that  we  expected  Marines  of  different 
competency  levels  to  have.  Over  the  course  of  the  studies,  we  adopted,  developed, 
and  refined  the  following  types  of  measures: 

•  Selected-response,  basic  rifle  marksmanship  knowledge  measure  to  broadly 
sample  the  domain  at  a  surface  level  (i.e.,  at  the  recognition  and  recall 
level). 


Constructed-response,  conceptual  understanding  (knowledge  map) 
instrument  that  was  intended  to  measure  Marines'  understanding  of  cause- 
effect  relations  among  different  factors  of  shooting. 
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•  Constructed-response,  shot-group-knowledge  instrument  that  was 
intended  to  measure  Marines'  knowledge  of  prototypical  shot  groups 
associated  with  different  shooter  errors. 

•  Constructed-response,  knowledge-of-proper-shooting-position  instrument 
that  was  intended  to  measure  Marines'  knowledge  of  proper  or  improper 
body  elements  of  different  shooter  positions. 

•  Surveys  of  worry  and  anxiety. 

•  Survey  items  covering  prior  record-fire  scores,  prior  training  experience, 
perceived  level  of  rifle  marksmanship  knowledge,  perceived  utility  of 
marksmanship  knowledge,  firing  line  experience,  and  prior  shooting 
experience. 


Overview  of  Studies  and  Findings 

Over  one  year,  five  data  collections  were  conducted  to  test  our  measures  on 
Marines  at  two  sites  that  conducted  rifle  marksmanship  training  (Stone  Bay,  NC, 
and  Quantico,  VA).  The  data  were  from  sustainment-level  (SLR)  Marines  and  entry- 
level  (ELR)  2nd  Lieutenants.  Table  4  presents  an  overview  of  the  studies  and  Table  5 
presents  an  overview  of  the  evidence  with  respect  to  each  of  the  types  of  validity 
evidence. 


Table  4 

Overview  of  Studies 


Date  and  study 

Location 

Sample 

Purpose 

June  2002,  Pilot  1 

Stone  Bay,  NC 

SLR 

Pilot  test  measures  (on  paper),  gather  feedback  from 
instructors. 

Nov.  2002,  Pilot  2 

Stone  Bay,  NC 

SLR 

Pilot  test  online  system  in  an  operational  environment. 

Dec.  2002,  Study  1 

Stone  Bay,  NC 

SLR 

Test  assessments  of  marksmanship  knowledge  and 
evaluate  prediction  of  qualification  score. 

Mar.  2003,  Study  2 

Quantico,  VA 

SLR 

Replicate  study  on  2nd  sample. 

May  2003,  Study  3 

Quantico,  VA 

ELR 

Replicate  study  on  2nd  LTs  undergoing  entry-level 
training. 
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significantly  and  moderately  related  to  shot  group  depiction  scores  and  perceived  utility 
of  marksmanship  knowledge. 

In  Study  3  (entry-level  2nd  LT  shooters),  expert-qualified  shooters  scored  significantly 
higher  on  basic  marksmanship  knowledge  and  perceived  level  of  marksmanship 
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We  had  no  expectations  of  the  degree  to  •  In  Study  2,  trait  worry  did  not  contribute  to  the  prediction  of  record-fire  score,  after  taking 

which  affective  variables  would  contribute  to  into  account  most  recent  record-fire  score,  firing  line  experience,  and  marksmanship  basic 
the  prediction  of  shooting  performance  knowledge. 

•  In  Study  3,  state  worry  contributed  to  the  prediction  of  record-fire  scores  (p  =  .07),  above 
and  beyond  firing  line  experience  and  aptitude. 


•  What  is  the  quality  of  the  validity  evidence  for  our  measures  of  rifle  marksmanship  knowledge? 

Extent  to  which  the  knowledge  measures  relate  to  *10  Pilot  Study  2,  basic  marksmanship  knowledge  was  related  low  but  significantly  to 
each  other  whether  participants  had  Phase  1  training  and  the  more  recently  they  had  Phase  1  training. 

•  In  Study  1,  low  to  moderate  correlations  were  found  between  all  measures  of  knowledge 
(basic  marksmanship  knowledge,  knowledge  map,  shot  group  depiction,  and  evaluation 
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SECTION  III:  PILOT  STUDIES 

Section  III  describes  two  pilot  studies  conducted  to  gather  information  about 
the  quality  of  our  assessments.  We  conducted  two  pilot  studies  and  three  studies  on 
different  samples.  The  purpose  of  the  first  pilot  study  was  to  gather  preliminary 
data  on  the  relationships  between  knowledge  of  rifle  marksmanship  and  shooting 
performance.  The  purpose  of  the  second  pilot  study  was  to  test  the  online  versions 
of  our  software  in  the  target  environment.  We  wanted  to  identify,  prior  to  the  main 
study,  any  technical  issues  as  well  as  gather  information  on  the  usability  of  our 
assessments. 


Pilot  Study  1  —  Method 

A  pilot  study  was  conducted  to  test  paper-based  versions  of  our  measures  and 
administration  procedures.  The  pilot  study  was  conducted  on  the  final  day  of  the 
Marines'  annual  qualification  week.  Our  focus  was  to  gather  information  on  the 
predictive  validity  of  our  measures  by  collecting  preliminary  data  on  the 
relationships  between  knowledge  of  rifle  marksmanship  and  shooting  performance. 

Participants 

Sixty  Marines  of  various  shooting  ability  participated  in  the  study.  In  general, 
the  Marines  were  between  the  ages  of  18-30. 

Design 

The  measures  were  administered  to  two  groups  of  Marines.  The  first  group 
was  Marines  who  had  been  classified  as  "unqualified"  at  least  once  during 
qualification  week.  The  second  group  was  Marines  who  had  qualified  at  either  the 
sharpshooter  or  expert  level.  The  tasks  were  timed  to  get  a  sense  of  the  amount  of 
time  needed  to  complete  the  task.  The  Marines  were  instructed  to  wait  for  further 
instructions  after  each  task  was  completed. 

Measures 

Record-fire  score.  The  main  outcome  measure  was  a  Marine's  record-fire  score. 
These  scores  were  self-reported  by  Marines. 
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Knowledge  mapping.  Knowledge  mapping  was  intended  to  measure  in 
general.  Marine's  knowledge  of  rifle  marksmanship  and  in  particular,  cause-effect 
relations  among  different  aspects  of  marksmanship.  The  task  required  Marines  to 
graphically  depict  their  understanding  of  rifle  marksmanship  in  terms  of  a  network. 
The  nodes  in  the  network  represented  concepts,  and  labeled  links  among  concepts 
represented  the  relationships  among  concepts.  The  sets  of  concepts  and  link  labels 
were  provided  to  Marines.  The  concepts  provided  were  Accuracy  of  aim  (AOA), 
Battlesight  zero  (BZO),  Bone  support  (BS),  Center  of  mass  (COM),  Eye  focus  on  front  sight 
post  (Eye  on  ESP),  Eye  relief  (ER),  Firm  hand  grip  on  trigger  (FHG),  Increased  target 
distance  (ITD),  Muscular  relaxation  (MR),  Rapid  fatigue  (RF),  Rifle  recoil  recovery  (RRR), 
Sight  alignment  (SA),  Sight  picture  (SP),  Sling  tension  (ST),  Stable  firing  position  (SFP), 
Stock  weld  placement  (SWP),  Trigger  control  (TC),  and  Weapon  movement  (WM).  The 
link  labels  provided  were  causes,  decreases,  distorts,  helps,  increases,  influences,  leads  to, 
and  requires. 

Basic  marksmanship  knowledge.  This  task  was  intended  to  serve  as  a  measure 
of  participants'  prior  knowledge  of  rifle  marksmanship.  Marines  were  given  20 
terms  and  asked  to  provide  a  short  written  response  that  could  be  a  definition  of  the 
term  or  the  significance  of  the  term.  The  terms  were  Weapon  conditions.  Weapons 
handling  safety  rules.  Remedial  action.  Aiming,  Breath  control.  Trigger  control.  Four 
shooting  positions.  Sight  alignment.  Sight  picture.  Zeroing,  BZO,  Natural  point  of  aim. 
Natural  respiratory  pause.  Muscling  the  rifle.  Bone  support.  Muscular  relaxation.  Front 
sight  post.  Rear  sight  elevation  knob.  Rear  sight  windage  knob,  and  Effects  of  weather. 

Shot-to-shot  explanation.  The  intent  of  this  task  was  to  measure  a  Marine's 
reasoning  from  shot  to  shot.  This  measure  was  an  attempt  to  mimic  the  cognitive 
demands  of  what  a  Marine  would  experience  on  the  firing  line  under  slow-fire 
conditions  (i.e.,  fire  a  round,  observe  strike,  evaluate  possible  causes  of  the  observed 
strike).  Five  frames  were  shown  to  the  Marine.  Each  successive  frame  showed 
successive  bullet  strikes.  The  Marine  was  required  to  check  off  possible  causes  that 
could  give  rise  to  the  observed  shot  pattern  from  a  list  of  18  possible  reasons.  An 
example  of  the  measure  and  hypothetical  response  are  contained  in  Appendix  D. 

Shot  group  pattern  analysis  task.  The  intent  of  this  task  was  to  measure  the 
extent  to  which  a  Marine  could  look  at  two  shot  patterns  and  posit  adjustments 
made  to  the  shooter  (position  or  rifle)  that  could  explain  the  difference  between  the 
two  shot  groups.  A  copy  of  the  measure  is  contained  in  Appendix  E. 
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Shot  group  depiction  task.  The  intent  of  this  task  was  to  measure  Marines' 
knowledge  of  shot  groups  associated  with  common  shooter  problems.  Marines  were 
required  to  draw  a  5-shot  group  for  problems  with  breathing,  sight  adjustment, 
trigger  control,  flinching,  bucking,  and  focusing  on  the  target.  A  sample  showing  the 
general  format  is  contained  in  Appendix  F. 

Background  survey.  The  intent  of  this  task  was  to  be  able  to  characterize  the 
sample  and  identify  background  characteristics  that  could  differentiate  high  from 
low  shooters.  The  kind  of  information  gathered  included  unit  information,  prior 
shooting  experience,  combat  status  of  unit,  marksmanship  training  experience,  and 
prior  qualification  scores. 

Procedure 

Our  original  intent  was  to  have  only  coaches  and  staff  review  our  measures; 
however,  an  opportunity  arose  to  have  Marines  undergoing  SLR  qualification  take 
our  measures,  although  it  was  outdoors  and  under  poor  conditions.  We  reasoned 
that  data  under  poor  conditions  could  still  provide  useful  information. 

Data  were  collected  across  two  groups  on  the  same  day.  The  first  group  of 
participants  had  shot  poorly  (i.e.,  unqualified  or  marksman).  The  second  group  was 
composed  of  participants  who  qualified  as  sharpshooter  or  expert.  Participants  were 
administered  the  knowledge  and  background  measures  as  shown  in  Table  6. 
Participants  were  instructed  to  take  as  much  time  needed  for  the  tasks.  In  general, 
participants  finished  all  the  tasks  within  the  allotted  time.  The  only  exception  was 
the  knowledge  map  task,  with  some  participants  taking  slightly  longer. 
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Table  6 

Administration  Schedule 

Time 

Task  allotted 

Introduction  to  study  5 

Basic  marksmanship  knowledge  task  15 

Shot  group  depiction  task  instructions  and  task  15 

Shot-to-shot  explanation  task  instructions  and  task  15 

Shot  group  pattern  analysis  task  10 

Background  survey  10 

Knowledge  mapping  instructions  and  task  20 


Pilot  Study  1— Results 

Analyses  of  our  preliminary  measures  suggested  a  relationship  between 
Marines'  self-reported  qualification  scores  and  knowledge  of  marksmanship.  Based 
on  this  particular  sample  of  Marines,  we  found  that  Marines'  self-reported 
qualification  score  was  related  to  the  quality  of  their  knowledge  map.  Self-reported 
qualification  scores  were  best  predicted  by  whether  the  Marine  took  the  coaches 
course  and  the  frequency  of  shooting  outside  of  their  USMC  duties  {R  =  .62,  p  <  .05), 
followed  by  the  extent  to  which  they  were  able  to  depict  shot  groups  associated  with 
common  shooter  errors  {R  =  .55,  p  <  .05).  Quality  of  the  knowledge  map  was  also 
related  to  qualification  scores,  although  marginally  statistically  significant  (r  =  .36, 
p  =  .08,  n  =  24).  Marines'  self-reported  previous  year's  qualification  score  was  also 
related  to  self-reported  qualification  score  (r  =  .38,  p  <  .05). 

Discussion  of  Pilot  Study  1 

Note  that  these  results  were  based  on  a  sample  of  Marines  who  were 
administered  paper  measures  under  very  poor  administration  conditions.  Marines 
were  sitting  on  bleachers  on  the  firing  range,  under  windy  conditions.  Also,  the 
measures  were  exploratory  and  our  first  attempt  at  measuring  knowledge  of  rifle 
marksmanship.  Thus,  these  results  are  at  best  only  suggestive  of  a  relationship 
between  shooting  skill  and  knowledge. 
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Pilot  Study  2—  Method 

The  main  objective  of  this  pilot  study  was  to  carry  out  a  technical  dry  run  of  the 
main  study  with  Marines  in  the  target  environment.  We  wanted  to  identify  any 
technical  issues  and  gather  information  on  the  usability  of  our  assessments.  We 
dropped  measures  from  Pilot  Study  1  that  did  not  appear  promising,  revised 
promising  measures,  and  converted  the  paper  measures  to  an  online  format.  During 
the  administration  of  our  measures,  numerous  technical  issues  arose.  The  most 
serious  issue  was  the  severe  network  security  restrictions,  which  prevented  the 
administration  of  our  QuickTime-based  assessments. 

Participants 

Ninety-five  Marines  of  various  shooting  ability  participated  in  the  study.  In 
general,  the  Marines  were  between  the  ages  of  18-30.  Ninety-three  of  the  participants 
were  men  and  2  were  women.  Sixty-six  point  seven  percent  of  the  participants  were 
White,  non-Hispanic;  11.1%  were  Latino/a;  9.1%  were  African-American;  and  4.0% 
were  Native  American.  Participants  had  completed  the  final  week  (Phase  II)  of  their 
annual  marksmanship  qualification. 

Measures 

Record-fire  score.  The  main  outcome  measure  was  participants'  record-fire 
score.  This  score  is  the  official  score  on  record  and  is  the  sum  of  subscores  on 
different  positions  at  different  distances.  The  various  positions  and  distances,  and 
score  ranges  are  shown  in  Table  7.  Cronbach's  alpha  for  the  record-fire  score  was 
.67.  Qualification  classifications  are  shown  in  Table  8.  Note  that  the  scale  differs  for 
scores  on  the  entry-level  and  sustainment-level  courses  of  fire.  A  conversion  table 
developed  by  the  USMC  is  given  in  Appendix  G. 
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Table  7 


Known-Distance  Course  Position  and  Scoring 


Position 

Distance 

(yards) 

Maximum 
possible  score 

Slow-Fire  Course 

Sitting 

200 

10 

Kneeling 

200 

10 

Kneeling 

300 

5 

Standing 

200 

10 

Prone 

500 

10 

Rapid-Fire  Course 

Kneeling 

200 

10 

Prone 

300 

10 

Table  8 

Qualification  Categories  and  Associated 
Score  Ranges 

Classification 

SLR  score  range 

Expert 

40-65 

Sharpshooter 

35-39 

Marksman 

25-34 

Unqualified 

<25 

Knowledge  mapping.  We  refined  the  knowledge  map  sets  of  terms  and  links 
and  developed  an  online  version  that  Marines  could  use  to  draw  their  maps.  The 
terms  were  7  factors  common  to  all  shooting  positions  (7F),  accuracy  (ACC),  aiming 
process  (AP),  bone  support  (BS),  breath  control  (BC),  elbow  placement  (EP),  eye  on  front 
sight  post,  (EFSP),  eye  relief  (ER),  finger  placement  (FP),  follow-through  (FT),  forward  hand 
placement  (FHP),  fundamentals  of  marksmanship  (FM),  grip  of  firing  hand  (GFH), 
muscular  relaxation  (MR),  muscular  tension  (MT),  natural  respiratory  pause  (NRP), 
placement  of  buttstock  in  shoulder  (PBS),  sight  adjustment  (SAD}),  sight  alignment 
(SALGN),  sight  picture  (SP),  stable  firing  position  (SFP),  stock  weld  placement  (SWP), 
trigger  control  (TC),  and  trigger  squeeze  (TSQ).  The  links  were  affects,  causes,  decreases. 
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during,  follows,  helps,  improves,  increases,  leads  to,  part  of,  prevents,  requires,  type  of,  uses, 
and  worsens. 

The  directions  for  the  task  were  given  orally  and  on  a  job  aid.  The  directions 
were: 

Make  a  knowledge  map  with  the  given  concepts  and  links.  The  concepts  are  related  to 
Rifle  Marksmanship.  Drag  the  concepts  to  the  screen.  Then  create  links  between  concepts 
to  show  how  the  concepts  are  related  to  each  other.  Make  as  many  relationships  you  can 
think  of  that  seem  important.  You  do  not  have  to  use  all  the  concepts  and  links. 

The  job  aid  also  included  the  lists  of  terms  and  links. 

A  screen  shot  of  the  computer  interface  of  the  Knowledge  Mapping  Tool  is 
shown  in  Figure  4.  The  task  required  Marines  to  create  a  knowledge  map  of  their 
understanding  of  rifle  marksmanship.  Participants  dragged  the  icon  labeled 
"concept"  onto  the  drawing  space.  When  the  mouse  was  released,  a  fixed  set  of 
terms  appeared  in  a  pop-up  menu  and  the  Marine  selected  the  term.  To  connect 
terms,  the  Marine  would  select  the  first  term,  hold  the  mouse  down  and  drag  to  the 
destination  term.  Upon  releasing  the  mouse,  a  fixed  set  of  links  would  appear  in  a 
pop-up  menu  and  the  Marine  would  select  the  desired  link. 
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Figure  4.  Sample  screen  shot  of  the  knowledge  map  task  computer  interface. 

Basic  marksmanship  knowledge.  We  converted  the  basic  marksmanship 
knowledge  measure  from  a  short-answer  format  to  a  selected-response  format.  We 
developed  items  for  the  areas  below  and  adopted  several  items  from  a  Stone 
Bay/Force  Service  Support  Group  instructor-developed  test.  The  measure  contained 
41  items  and  covered  sight  picture,  sight  adjustment,  sight  alignment,  weapons 
safety,  breathing,  trigger  control,  stock  weld,  eye  relief,  bone  support,  firing  hand 
placement,  follow-through,  muscle  relaxation,  forward  hand  placement,  grip  of 
firing  hand,  and  muscular  tension.  Cronbach's  alpha  for  the  measure  was  .85. 

Shot  group  depiction  task.  Similar  to  Pilot  Test  1,  we  administered  a  slightly 
revised  version  of  this  measure.  The  task  was  designed  to  measure  Marines' 
knowledge  of  shot  groups  associated  with  common  shooter  problems.  Marines  were 
required  to  draw  a  5-shot  group  for  problems  with  breathing,  sight  adjustment, 
flinching,  bucking,  and  focusing  on  the  target.  Cronbach's  alpha  for  the  measure 
was  .26. 
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The  directions  for  the  task  were  given  orally  and  via  a  job  aid.  The  directions 
were: 

Each  icon  represents  a  particular  shooter  error.  For  each  type  of  error,  plot  five  bullet 

strikes  that  reflect  the  conditions  at  300  yards  of  a  right-handed  shooter  in  the  kneeling 

position.  Unless  otherwise  stated,  assume  no  wind/ weather  effects. 

Note: 

1.  Use  exactly  5  shots  per  error  type. 

2.  Plot  all  shots  on  the  same  target.  There  is  only  one  target.  Each  error  type  is 
indicated  with  a  different  color. 

3.  The  table  below  provides  a  more  detailed  description  of  the  type  of  problem. 

A  screen  shot  of  the  computer  interface  is  shown  in  Figure  5.  The  task  required 
Marines  to  drag  exactly  5  icons  of  each  color  from  the  left  column  onto  the  target  to 
show  the  shot  pattern.  Each  icon  was  colored  and  represented  a  different  kind  of 
shooter  error.  The  types  of  errors  are  shown  in  Table  9. 
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Figure  5.  Sample  screen  shot  of  the  shot  group  depiction  task  computer  interface. 


Table  9 

Types  of  Errors  for  the  Shot  Group  Depiction  Task 
Icon  color  Type  of  problem 

Gray  Breathing  while  firing 

Blue  Aiming  eye  focused  on  the  target  instead  of  the  front  sight  post  while  firing 

Green  Flinching  the  body  while  firing  (sudden  small  backward  movement  of  body) 

Red  Bucking  the  shoulder  into  the  rifle  stock  while  firing 

Yellow  Sight  adjustment  problem  (inadequate  compensation)  with  wind  blowing  from  the 

right 


Participants'  shot  group  depictions  were  scored  automatically  using  an 
algorithm  that  was  based  on  an  expert's  representation  (see  Appendix  H).  Each  type 
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of  shot  group  was  scored  as  correct  or  incorrect.  To  verify  the  automated  scoring 
performance,  shot  group  scores  produced  by  the  automated  scoring  algorithm  were 
compared  to  scores  assigned  by  a  rater  following  a  rubric  (Appendix  I).  The  rubric 
was  developed  from  descriptions  provided  by  USMC  marksmanship  coaches  and 
from  reviews  of  shot  group  analyses  in  U.S.  Army  and  USMC  field  manuals  (U.S. 
Army,  1989;  USMC,  1992). 

Agreement  between  the  automated  scoring  routine  and  the  human  rater 
ranged  from  a  low  of  88.5%  for  the  problems  of  target  focus  and  sight  adjustment, 
and  a  high  of  96.5%  for  flinching. 

Table  10 


Percent  Agreement  Between  Human 
Rater  and  Automated  Scoring  System 
(N  =  113) 


Type 

Percent 

Breathing 

91.2 

Target  focus 

88.5 

Flinching 

96.5 

Bucking 

94.7 

Sight  adjustment 

88.5 

Note.  Sample  included  data  from  Pilot 
Study  1. 


Evaluation  of  shooter  positions  (ESP).  This  task  was  intended  to  measure  a 
Marine's  skill  at  identifying  proper  and  improper  firing  positions  of  a  shooter 
posing  in  proper  and  improper  positions.  The  shooter  was  shown  in  QuickTime  VR, 
and  Marines  could  rotate  the  image  to  view  the  shooter  from  different  angles. 
Marines  were  asked  to  judge  whether  the  shooter's  position  was  proper  or  improper 
on  the  following  elements:  placement  of  firing  hand,  placement  of  forward  hand, 
elbow  placement,  stock  weld,  breath  control,  rifle  butt  in  pocket  of  shoulder,  leg 
placement,  feet  placement,  body  placement,  and  overall  position. 
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The  directions  for  the  task  were: 

Please  evaluate  the  Marine's  application  of  rifle  marksmanship  skills.  You  can  examine 
the  Marine  in  the  following  ways— 

1.  Rotating  the  picture  to  allow  you  to  examine  the  Marine  from  all  directions. 

2.  Clicking  on  "hotspots"  on  the  picture— close-up  pictures  are  available  for  certain 
areas  of  the  Marine. 

3.  Viewing  a  5-second  animation  of  the  sight  picture  preceding  the  firing  of  the  shot. 

4.  Viewing  the  strike  of  the  round  on  the  target. 

You  will  be  asked  to  evaluate  how  proper  the  Marine's  position  is  on  the  following: 

Placement  of  firing  hand 
Placement  of  forward  hand 
Elbow  placement 
Stock  weld 
Breath  control 

Rifle  butt  in  pocket  of  shoulder 
Leg  placement 
Feet  placement 
Body  placement 

In  addition,  you  will  be  asked  for  an  overall  judgment  of  the  Marine's  application  of  the 
fundamentals. 

You  are  to  use  a  4-point  scale  ranging  from  "proper"  to  "improper." 

A  screen  shot  of  the  computer  interface  is  shown  in  Figure  6.  The  task  required 
participants  to  diagnose  possible  problems  with  a  shooter's  firing  position.  The 
participant  was  presented  with  a  shooter  (upper  left  quadrant  of  Figure  6).  The 
figure  was  a  QuickTime  VR  image,  so  the  participant  could  rotate  the  image  and 
have  a  360-degree  view,  as  well  as  view  the  shooter  from  different  angles  (i.e.,  as  if 
viewing  from  ground  level  or  slightly  above).  The  participant  could  also  view  5 
seconds  of  the  sight  picture  immediately  prior  to  firing  the  round  (upper  right 
quadrant  of  Figure  6).  In  addition,  there  were  "hotspots"  on  the  shooter  that  could 
be  viewed  close  up.  The  available  hotspots  were  of  the  different  body  elements 
important  to  shooting  (e.g.,  feet,  elbows,  stock  weld,  firing  hand,  grip  — shown  as 
ovals  in  the  upper  left  pane  in  Figure  6).  If  the  participant  clicked  on  a  hotspot  area 
of  the  shooter,  a  close-up  image  would  appear  (lower  left  quadrant  of  Figure  6). 
Finally,  the  participant  was  required  to  judge  the  extent  to  which  each  body  element 
was  in  its  proper  position. 
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Figure  6.  Sample  screen  shot  of  the  ESP  task  computer  interface. 

Self-regulation  survey.  This  measure  was  adopted  from  O'Neil  and  Herl 
(1998).  The  intent  of  this  task  was  to  measure  Marines'  use  of  self-regulation 
strategies  with  respect  to  classroom-based  learning  (i.e.,  planning,  self-checking). 
Cronbach's  alpha  for  the  planning  and  self-checking  scales  were  .85  and  .83, 
respectively. 

Background  survey.  We  administered  a  revised  background  survey  that  was 
similar  to  Pilot  Study  1.  The  revisions  included  improved  terminology  and  more 
coverage  in  areas  that  looked  promising  from  Pilot  Study  1.  In  particular,  we 
gathered  more  detailed  information  on  Marines'  unit,  rank,  marksmanship  coaches 
course  history,  and  shooting  experience. 

Procedure 

Data  collection  was  conducted  across  two  sessions  (morning  and  afternoon)  on 
the  same  day.  We  administered  the  measures  in  two  formats,  paper  and  online.  This 
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allowed  us  to  check  for  potential  format  differences.  For  the  online  versions  of  our 
measures,  researchers  from  CRESST  demonstrated  the  use  of  the  software.  The 
paper  versions  were  self-explanatory.  Because  of  firewall  restrictions  on  MarineNet, 
the  ESP  measure  could  not  be  administered.  We  administered  the  measures  in  the 
order  and  allotted  times  shown  in  Table  11. 


Table  11 

Administration  Schedule 


Task 

Time 

allotted 

Introduction  to  study 

5 

Knowledge  mapping  instructions 

10 

Knowledge  mapping  task 

20 

Self-regulation  survey 

5 

Shot  group  depiction  task  instructions  and  task 

10 

Basic  marksmanship  knowledge  task 

20 

Background  survey 

5 

Evaluation  of  shooter  positions  (planned  but  did 
not  occur  due  to  firewall  restrictions) 

45 

Pilot  Study  2  —  Results 


Preliminary  Analyses 

During  the  morning  session  the  system  response  was  very  sluggish  and  many 
Marines  experienced  "page  not  found"  errors.  We  determined  that  these  errors  were 
primarily  due  to  a  very  slow  network  connection.  We  fixed  the  problem  prior  to  the 
afternoon  session. 

We  conducted  three  2(morning,  afternoon)  x  2(paper,  online)  ANOVAs  on  the 
knowledge  mapping,  shot  group  depiction,  and  basic  marksmanship  knowledge 
scores,  to  check  for  differences  by  session  and  format.  Session  was  included  as  a 
factor  given  the  technical  problems  in  the  morning.  No  significant  effects  were 
found  for  the  knowledge  mapping  and  shot  group  depiction;  however,  a  session  x 
format  effect  was  found  for  the  basic  marksmanship  knowledge  measure,  F(l,  94)  = 
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9.25,  MSE  =  317.4,  p  <  .05.  Follow-up  pairwise  comparisons  showed  that  for  the 
morning  session.  Marines  administered  the  paper  measure  (M  =  27.0,  SD  =  4.5,  n  = 
17)  performed  significantly  higher  than  those  administered  the  online  measure  (M  = 
20.3,  SD  =  6.7,  n  =  32).  No  format  differences  were  found  for  the  afternoon  session. 

We  concluded  that  the  most  likely  cause  for  the  format  differences  in  the 
morning  session  was  the  computer  problems.  Given  that  there  were  no  other  format 
differences  on  our  other  measures  (particularly  the  knowledge  mapping  scores),  we 
concluded  that  format  was  not  likely  to  have  a  large  effect;  thus,  we  pooled  the 
sample  across  format. 

Relationship  of  Measures  to  Record-Fire  Score 

In  terms  of  relationships  between  our  assessments  and  qualification  score,  we 
again  found  relationships  that  suggested  a  link  between  knowledge  of  rifle 
marksmanship  and  self-reported  qualification  score.  For  example.  Marines'  self- 
reported  qualification  score  was  related  to  their  performance  on  our  basic 
marksmanship  knowledge  measure  (r  =  .21,  p  <  .05),  if  they  rated  their  job  as  non¬ 
ground  combat  (r  =  .29,  p  <  .01),  the  number  of  months  since  their  last  Phase  I 
training  (r  =  -.25,  p  <  .05),  and  their  self-reported  most  recent  qualification  score  (r  = 
.36,  p  <  .01). 

With  respect  to  performance  on  our  assessments,  we  found  that  Marines  scored 
higher  on  our  basic  marksmanship  knowledge  measure  (a)  if  they  had  Phase  I 
training  (r  =  .20,  p  <  .05);  (b)  the  more  recently  (in  months)  they  had  Phase  I  training 
(r  =  .24,  p  <  .05);  and  (c)  the  higher  their  self-reported  most  recent  qualification  score 
(r  =  .23,  p  <  .05).  Also,  Marines  tended  to  perform  higher  on  the  shot  group  depiction 
task  (a)  if  they  rated  their  job  as  non-ground  combat  (r  =  .21,  p  <  .05);  and  (b)  the 
more  hours  they  reported  shooting  as  part  of  their  USMC  duties  (r  =  .30,  p  <  .05). 

Prediction  of  Qualification  Score 

The  best  predictor  of  self-reported  qualification  scores  was  from  the  following 
set  of  variables:  (a)  average  of  scores  on  the  plarming  and  checking  scales,  (b) 
number  of  months  since  the  Marine's  last  Phase  I  training,  (c)  score  on  the  basic 
marksmanship  knowledge  measure,  (d)  self-reported  most  recent  qualification 
score,  and  (e)  combat  status  of  the  Marine's  job.  The  resulting  multiple  R  for  this 
sample  was  .53. 
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Discussion  of  Pilot  Study  2 

As  with  the  first  pilot  study,  the  results  related  to  our  assessments  need  to  be 
interpreted  with  caution.  These  data  are  from  Marines  who  took  our  measures 
online  and  on  paper.  Further,  about  half  of  the  sample  who  received  the  online 
assessments  experienced  technical  problems  with  the  software  (e.g.,  slow-loading  or 
non-loading  Web  pages;  very  slow  system  performance). 

Given  this  caution,  the  results  again  suggest  a  knowledge  component  to 
shooting  performance.  As  in  the  first  pilot  study,  self-reported  qualification  scores 
were  related  significantly  to  measures  of  knowledge  of  the  fundamentals,  although 
the  magnitude  of  the  relationship  was  low.  Evidence  that  our  basic  marksmanship 
knowledge  measure  was  tapping  knowledge  of  rifle  marksmanship  is  seen  in  the 
relationship  with  the  Phase  I  training  variables  (i.e.,  higher  scores  on  the  basic 
marksmanship  knowledge  measure  if  they  had  Phase  1  training  and  the  more  recent 
the  Phase  I  training).  Finally,  the  regression  equation  also  supports  the 
interpretation  that  there  is  a  knowledge  component  to  shooting  performance.  Basic 
marksmanship  knowledge  scores  and  recency  of  Phase  I  training  improve  the 
predictability  of  shooting  performance  above  experience  variables  alone. 

SECTION  IV:  MAIN  STUDIES 
Study  1  —  Method 

The  first  study  conducted  in  December  2002  was  intended  to  examine  the 
extent  to  which  our  online  assessments  could  predict  shooting  performance  among 
sustainment-level  Marines.  The  full  set  of  measures  was  administered  online,  and 
the  current  and  most  recent  qualification  scores  were  gathered  from  the  Stone  Bay 
database,  not  Marines'  self-reports.  UCLA  researchers  also  observed  the  pit 
verification  process  and  Stone  Bay  also  provided  verifiers  to  ensure  the  Marines' 
bullet  strikes  were  accurately  recorded  on  the  score  cards. 

Participants 

One-hundred  fifty-nine  Marines  of  various  shooting  skill  participated  in  the 
study.  A  description  of  the  sample  is  shown  in  Table  12  through  Table  15.  In  general, 
the  participants  were  male  (94%),  enlisted,  with  less  than  2.5  years  in  service,  and 
from  a  mix  of  support,  base,  and  combat  units. 
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Table  12 

Descriptive  Statistics  on  Background  Variables 


Variable 

n 

M 

SD 

Min. 

Max. 

Most  recent  qualification  score 

159 

40.2 

7.9 

25 

58 

Second  most  recent  qualification  score 

119 

36.9 

8.0 

24 

54 

Third  most  recent  qualification  score 

72 

39.9 

9.3 

24 

65 

Age 

159 

21.9 

3.2 

19 

45 

Years  in  service 

159 

2.5 

2.7 

0 

30 

Frequency  of  shooting  as  part  of  duties^ 

159 

2.1 

1.0 

1 

5 

Frequency  of  shooting  outside  of  duties® 

159 

2.1 

1.4 

1 

5 

Years  of  shooting  experience  before  Marines 

159 

3.2 

4.7 

0 

19 

®1  =  Never;  2  =  Once  or  twice;  3  =  A  few  times;  4 

=  Often;  5  = 

Very  often. 

Table  13 

Distribution  of  Participants  by  Combat  Status 

Frequency 

Percent 

Division® 

51 

31.7 

Force  Service  Support  Group'’ 

44 

27.3 

Support/ Base’’ 

39 

24.2 

Other’' 

27 

16.8 

«MEU,  2/10,  2/6, 2nd  Marine  Div,  2nd  MEB,  3/2, 3/8,  3/6, 4 
MEB,  8th  Marine  Regiment,  II MEF.  bESSC.  '^Base  and  formal 
schools.  ^Aviation,  division  staff.  Marine  security  guard. 


Table  14 

Distribution  of  Participants  by  Rank 


Frequency  Percent 


Private  (E-1) 

2 

1.3 

Private  First  Class  (E-2) 

11 

6.9 

Lance  Corporal  (E-3) 

105 

66.0 

Corporal  (E-4) 

34 

21.4 

Sergeant  (E-5) 

7 

4.4 
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Table  15 


Distribution  of  Participants  by  Marksmanship  Instructor  Training 


Course 

No.  of  participants  who 

completed  course 

Marksmanship  Coaches  Course  (MOS  8530) 

11 

Marksmanship  Instructor  (MOS  8531) 

4 

Small  Arms  Weapons  Instructor  (MOS  8532) 

1 

Range  Officer  (MOS  9925) 

0 

Design 

The  tasks  were  administered  to  six  groups  over  three  days  during  Phase  I  and 
Phase  II.  Seventy-nine  participants  had  attempted  qualification  and  13  had  not 
attempted  qualification.  The  remaining  number  of  participants  did  not  respond.  The 
online  delivery  problems  were  resolved  and  the  online  administration  during  the 
data  collection  was  problem  free.  Training  on  the  assessment  tasks  was  standardized 
using  an  online  video  training  task  to  ensure  that  each  participant  had  the  same 
training.  Job  aids  were  also  provided  to  each  participant. 

Measures 

The  following  measures  were  provided  to  us  by  the  USMC  or  administered  to 
Marines: 

Record-fire  score.  The  main  outcome  measure  was  Marines'  record-fire  score. 
These  scores  were  the  official  score  of  record  from  the  USMC  database. 

Most  recent  record-fire  scores.  These  were  self-reported  qualification  scores  for 
the  three  most  recent  qualification  trials. 

Knowledge  mapping.  We  used  the  same  measure  from  Pilot  Study  2. 

Scoring  of  participant  knowledge  maps  was  conducted  by  comparing 
participants'  knowledge  maps  against  a  criterion  map.  The  criterion  map  was 
generated  by  our  subject  matter  expert,  who  had  completed  the  coaches  course  and 
the  scout  sniper  program.  A  participant's  score  was  the  count  of  the  number  of 
propositions  in  his  or  her  map  that  were  also  in  the  criterion  map.  The  criterion  map 


# 


Knowledge,  Models  and  Tools  to  Improve  the  Effectiveness  of  Naval  Distance  Learning  47 

is  given  in  Appendix  J  (Expert  Nov./Dec.,  screen  shots)  and  Appendix  K  (Expert 
Nov./ Dec.,  propositions). 

Basic  marksmanship  knowledge.  We  used  the  same  measure  from  Pilot  Study 
2  with  slight  modifications  to  wording. 

Shot  group  depiction  task.  We  used  the  same  measure  from  Pilot  Study  2. 

Self-regulation  survey.  We  used  the  same  measure  from  Pilot  Study  2. 

Background  survey.  We  used  the  same  measure  from  Pilot  Study  2. 

Evaluation  of  shooter  positions  (ESP).  We  used  the  same  measure  from  Pilot 
Study  2. 

Procedure 

We  administered  the  measures  in  two  formats,  paper  and  online.  Most  of  the 
participants  received  the  assessments  online;  however,  because  there  were 
additional  participants  available  (i.e.,  participants  were  provided  at  the  company 
level,  and  we  did  not  have  enough  computers;  thus,  these  additional  participants 
were  administered  paper  versions  of  our  measures).  We  followed  the  same 
administration  schedule  and  procedure  as  Pilot  Study  2,  with  the  exception  that  the 
instructions  for  each  online  task  were  delivered  via  an  online  training  demo. 

Study  1 — Results 

We  present  the  results  of  this  study  around  two  key  issues:  (a)  the  prediction  of 
record-fire  performance,  and  (b)  the  reliability  and  the  validity  evidence  of  cognitive 
measures  used  in  this  study.  Our  research  questions  around  the  two  main  issues  are 
repeated  below. 

•  To  what  extent  can  record-fire  performance  be  predicted  from  knowledge, 
perceptual-motor,  and  affective  variables? 

•  What  combination  of  perceptual-motor,  cognitive,  and  affective  measures 
yield  the  best  predictors  of  record-fire  performance? 

•  What  is  the  quality  of  the  validity  evidence  for  our  measures  of  rifle 
marksmanship  knowledge? 

The  first  two  questions  address  the  first  issue  of  prediction,  and  the  last 
question  addresses  the  strength  of  the  validity  evidence  for  our  measures. 
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Prediction  of  Record-Fire  Performance 

To  predict  record-fire  performance,  we  conducted  a  series  of  multiple 
regression  analyses.  Predictions  were  made  for  the  whole  group,  which  draws  from 
the  entire  sample,  and  subgroups.  In  this  section,  we  present  the  results  of  the 
whole-group  prediction  first,  followed  by  four  subgroup  predictions,  for  the  same 
set  of  questions. 

Prediction  of  Record-Fire  Performance  Based  on  the  Entire  Sample 

To  predict  the  record-fire  performance  of  Marines,  we  conducted  multiple 
regression  analyses  for  the  entire  sample  {N  =  103).  Multiple  regression  analyses  can 
be  used  to  determine  which  combination  of  measures  is  the  best  predictor  of 
performance  and  the  quality  of  the  prediction,  when  the  ranges  of  important 
variables  are  naturally  occurring  and  not  restricted  (this  advantage  will  be  clear 
when  compared  to  the  subsample  prediction  in  the  next  section). 

The  outcome  variable  of  multiple  regression  is  record-fire  score.  Table  16 
presents  descriptive  statistics  and  intercorrelations  for  the  measures,  and  Table  17 
presents  the  descriptive  statistics  for  the  measures  included  in  the  multiple 
regression  analysis.  Table  18  displays  the  results  of  the  multiple  regression  analysis. 
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Table  17 

Descriptive  Statistics  for  Variables  Used  in  the  Multiple  Regression  Analysis 

Variable 

n 

M 

SD 

Min. 

Max. 

Record-fire  score 

156 

39.92 

9.41 

1 

58 

Most  recent  record-fire  score 

121 

35.41 

8.78 

6 

56 

Basic  marksmanship  knowledge 

161 

26.47 

6.17 

0 

40 

Shot  group  depiction 

154 

1.53 

1.13 

0 

4 

Evaluation  of  shooter  positions 

160 

59.88 

8.32 

34 

79 

Knowledge  map 

151 

2.28 

2.27 

0 

11 

Whether  took  coaches  course 

161 

0.07 

0.26 

0 

1 

Mean  of  self-regulation  planning  and 
checking  scales 

156 

3.18 

0.52 

1.44 

4.00 

Perceived  utility  of  marksmanship  knowledge 
to  shooting  performance 

157 

4.43 

0.90 

1 

5 

No,  of  months  since  last  Phase  I  training 

152 

4.45 

7.07 

0 

26 

Table  18 


Regression  Summary  Predicting  Record-Fire  Performance 


Variable 

Coefficient 

SE 

p  value 

Most  recent  record-fire  score 

0.36 

0.11 

.00 

Basic  marksmanship  knowledge 

0.34 

0.19 

.08 

Shot  group  depiction 

0.34 

0.83 

.68 

Evaluation  of  shooter  positions 

0.19 

0.11 

.09 

Knowledge  map 

-0.08 

0.44 

.86 

Whether  took  coaches  course 

3.57 

2.85 

.21 

Mean  of  self-regulation  planning  and  checking  scales 

1.72 

1.75 

.33 

Perceived  utility  of  marksmanship  knowledge  to 
shooting  performance 

0.36 

1.06 

.74 

No.  of  months  since  last  Phase  I  training 

-0.24 

0.13 

.07 
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What  combinations  of  perceptual-motor  (proxies),  cognitive,  and  affective 
measures  yield  the  best  (practical)  predictors  of  record-fire  performance?  As 
shown  in  Table  18,  basic  marksmanship  knowledge,  shot  group  knowledge,  proper 
position  identification,  knowledge  map,  self-regulation,  and  perceived  value  of 
knowledge  to  shooting  performance  showed  up  as  good  predictors  among  cognitive 
measures,  while  most  recent  record-fire  score,  shooting  coach  status,  and  the 
number  of  months  since  last  sustainment-level  training  on  marksmanship  made  a 
good  set  of  predictors  among  copious  background  measures. 

This  combination  of  cognitive  and  background  measures  was  chosen  among  all 
other  possible  combinations,  based  mainly  on  theoretical  considerations,  although  it 
was  modified  based  on  empirical  results.  Specifically,  sets  of  covariates  were  chosen 
first  according  to  the  theoretical  framework,  and  those  were  modified  by  empirical 
results,  such  as  bi-variate  correlation  between  the  possible  predictor  and  record-fire 
performance,  and  the  multiple  correlation  coefficient  (R)  of  multiple  regression 
analyses.  For  example,  variables  such  as  the  number  of  months  since  last 
sustainment-level  training  on  marksmanship  were  discovered  empirically.  In 
addition,  some  variables  that  were  not  statistically  significant  were  retained  in  the 
regression  out  of  theoretical  considerations. 

The  best  predictor  of  record-fire  score  was  the  most  recent  record-fire  score, 
which  was  the  single  significant  predictor  in  the  regression  {p  <  .01).  Basic 
marksmanship  knowledge,  proper  position  identification,  and  the  number  of 
months  since  last  sustainment-level  training  on  marksmanship  were  also  close  to 
statistical  significance  with  p  values  between  .05  and  .10.  Participants  tended  to  fire 
more  accurately  by  3.2  points  when  their  most  recent  record-fire  score  was  1 
standard  deviation  (8.7  points)  higher,  holding  constant  all  the  other  variables  in  the 
equation.  Again  holding  constant  all  the  other  variables,  participants  are  expected  to 
fire  more  accurately  by  2.1  and  1.6  points  when  they  are  1  standard  deviation  higher 
in  the  basic  marksmanship  knowledge  and  proper  position  identification  measures, 
respectively  (6.2  points  and  1.1  points);  a  participant  tended  to  fire  more  accurately 
by  1.7  points  when  the  last  sustainment-level  training  on  marksmanship  was  within 
the  last  7  months  (which  is  1  standard  deviation  of  the  variable). 

To  what  extent  can  record-fire  performance  be  predicted  from  the 
combinations  of  perceptual-motor  (proxies),  cognitive,  and  affective  measures? 
Two  criteria  were  used  to  evaluate  the  quality  of  the  prediction  model.  The  first  is 
the  multiple  correlation  (R)  or  the  square  of  the  multiple  correlation  (R^).  The  current 
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multiple  regression  yielded  R  of  .56  and  of  .31.  Thirty-one  percent  of  the  total 
variability  in  the  record-fire  performance  was  accounted  for  by  the  combination  of 
predictors  in  the  regression.  This  is  only  a  moderate  level  of  prediction  given  a  set  of 
nine  carefully  selected  predictors,  as  the  other  69%  still  remained  unexplained. 

The  95%  confidence  interval  of  predicted  record-fire  performance  scores  is  the 
second  criterion.  Figure  7  displays  the  confidence  intervals  for  all  105  participants 
analyzed  in  the  regression.  The  length  of  interval  varied  across  participants,  the 
average  being  around  10;  the  smallest  interval  was  4.98  and  the  largest  was  17.72. 
Some  confidence  intervals  are  fairly  wide,  suggesting  that  the  level  of  prediction 
needs  to  be  more  precise  for  some  participants. 

One  of  the  most  conservative  ways  to  assess  participants  in  this  framework  is 
to  disqualify  participants  who  have  the  lowest  value  of  the  interval  below  25  (this 
cutoff  is  based  on  USMC  qualification  categories).  This  results  in  disqualifying  5  of 
the  105  participants.  The  record-fire  performance  score  of  the  5  participants  were  1, 
27,  31,  35,  and  40.  In  addition,  4  participants  who  had  record-fire  performance  scores 
of  less  than  25  were  not  identified  as  unqualified.  To  better  serve  the  classification 
purpose,  more  precise  prediction  is  necessary. 


Figure  7.  The  95%  confidence  intervals  of  predicted  values  from  a 
regression  analysis. 
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Prediction  of  Record-Fire  Performance  Based  on  the  Subsamples 

In  addition  to  analyzing  the  entire  sample,  we  attempted  to  create  subgroups  of 
the  sample  and  make  predictions  separately  for  each  subgroup.  The  rationale  behind 
the  subgroup  analyses  comes  from  the  skill  acquisition  model.  With  a  new  skill, 
performance  evolves  over  different  stages  of  development.  Presumably,  the  different 
stages  are  characterized  by  distinct  cognitive  and  performance  characteristics  (e.g., 
Ackerman,  1987,  1992).  From  a  regression  perspective,  this  could  allude  to 
qualitatively  different  subgroups,  each  of  which  possibly  have  different  sets  of 
predictors  or  different  relationships  between  record-fire  performance  and  a  set  of 
predictors. 

Four  subgroups  (high  performers,  medium  performers,  low  performers,  and 
learners)  were  created  using  two  descriptive  statistics:  the  average  and  the  standard 
deviation  of  record-fire  scores  across  two  to  four  occasions  (the  available  number  of 
record-fire  scores  ranged  from  two  to  four  for  individual  participants).  The  average 
of  two  to  four  record-fire  scores  indicates  an  average  level  of  performance  of  an 
individual  participant,  whereas  the  standard  deviation  is  indicative  of  the  variability 
of  the  performance  of  a  participant  across  different  occasions.  Note  that  the  average 
and  the  standard  deviation  here  are  not  across  participants  but  across  occasions 
within  each  participant.  The  criteria  used  for  classifying  participants  was:  learners, 
M  >  30,  SD  >  10;  high  performers,  M  <  45,  SD  <  10;  medium  performers,  30  <  M  <  45, 
SD  <  10;  and  low  performers,  M  <  30. 
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Figure  8.  Subgroup  classification  based  on  a  scatterplot  of  spread  and  mean 
of  record-fire  scores  across  occasions. 


Figure  8  displays  the  classification  of  the  four  groups  based  on  the  two 
descriptive  statistics.  The  learner  group  was  separated  out  because  the  variability 
across  occasions  was  apparently  greater  than  the  rest  of  the  sample  (the  standard 
deviation  was  higher  than  10).  The  variability  of  these  participants  normally  comes 
from  a  low  score  in  previous  occasions  and  a  high  score  in  later  occasions,  with  the 
exception  of  one  case.  We  interpreted  that  participants  in  this  group  switched  from  a 
fairly  low  level,  possibly  the  stage  of  novice,  to  a  fairly  high  level  in  a  relatively  short 
period  of  time. 

The  identification  of  the  other  groups  was  based  more  on  average  level  of 
performance  (i.e.,  the  average  of  record-fire  scores),  after  separating  out  the  learner 
group.  However,  the  two  cut-points  that  classify  the  three  groups,  30  and  45,  were 
determined  making  use  of  the  standard  deviation,  as  shown  in  Figure  8.  The  average 
scores  of  the  high  spread  group  (learners)  begin  at  30  and  end  at  45.  The  reasoning 
behind  this  was  that  high  and  low  performers  would  be  relatively  stable  in  their 
performance  across  occasions.  This  classification  is  very  sample-dependent  and  the 
assumption  is  open  to  more  investigation. 

Table  19  presents  the  descriptive  statistics  of  final  record-fire  performance  by 
subgroup.  According  to  USMC  qualification  categories,  record-fire  scores  below  25 
indicate  that  a  Marine  is  unqualified,  scores  of  25  to  34  indicate  marksman,  scores  of 
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35  to  39  indicate  sharpshooter,  and  scores  of  40  to  65  indicate  expert.  This 
categorization  is  more  detailed  in  that  the  cutoffs  are  on  a  finer  scale  and  show  a 
desirable  level  of  performance  (above  35)  and  a  undesirable  level  of  performance 
(below  25)  in  an  absolute  sense.  However,  it  does  not  consider  the  dynamics  of  rifle 
marksmanship  as  a  learned  skill. 

The  comparison  between  the  USMC  qualification  categories  and  the  data- 
driven  classification  shown  in  Table  19  might  be  informative.  Low  performers  were 
mostly  unqualified  and  marksmen.  Medium  performers  were  mostly  marksmen  and 
sharpshooters,  including  some  experts.  High  performers  were  all  experts.  Learners 
were  on  average  close  to  high  performers  but  were  more  heterogeneous,  covering 
marksmen,  sharpshooters,  and  experts. 

Table  19 


Descriptive  Statistics  of  Record-Fire  Performance  by  Subgroup 


n 

M 

Min. 

Max. 

Entire  sample 

156 

39.92 

1 

58 

Learners 

17 

47.24 

28 

54 

Low  performers 

19 

25.32 

1 

40 

Medium  performers 

98 

39.93 

25 

54 

High  performers 

16 

49.94 

40 

58 

What  combinations  of  perceptual-motor  (proxies),  cognitive,  and  affective 
measures  yield  the  best  (practical)  predictors  of  record-fire  performance?  As 
expected  from  the  theory  of  skill  learning,  different  sets  of  predictors  and  different 
relationships  showed  up  in  four  different  subgroups.  Table  20  summarizes  which 
cognitive  and  background  measures  made  a  good  set  of  predictors  in  each  of  the 
four  groups.  Table  21,  Table  22,  Table  23,  and  Table  24  present  the  results  of  multiple 
regression  analyses  for  the  four  groups  in  the  order  of  learners,  high  performers, 
medium  performers,  and  low  performers. 

For  the  groups  of  learners  and  high  performers,  an  additional  multiple 
regression  analysis  was  conducted  without  one  observation  (so-called  leave-one-out 
analysis).  In  the  high  performer  group,  one  observation  was  identified  using  the 
largest  absolute  value  of  the  studentized  residual  (-2.97).  In  the  learner  group,  one 
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observation  was  identified  for  a  substantive  reason.  One  observation  had  a  high 
previous  performance  and  a  low  later  performance,  while  all  the  other  observations 
had  a  low  previous  performance  and  a  high  later  performance. 


Table  20 


Regression  Model  Variables  by  Type  of  Analysis  and  Type  of  Measure 


Analysis  method 

Cognitive  measures 

Background  measures 

Entire  sample 

1. 

Basic  marksmanship  knowledge 

1. 

Most  recent  record-fire  score 

2. 

Shot  group  depiction 

2. 

Whether  a  shooting  coach 

3. 

Proper  position  identification 

3. 

Months  since  last  sustainment- 
level  training  on  marksmanship 

4. 

Knowledge  map 

5. 

Self-regulation  (classroom 
training) 

6. 

Perceived  utility  of 
marksmanship  knowledge  to 
shooting  performance 

By  subsample 

Learners 

1, 

Basic  marksmanship  knowledge 

1. 

Number  of  years  in  service 

2. 

Shot  group  depiction 

2. 

Number  of  years  of  shooting 
experience  prior  to  joining  USMC 

High  performers 

1. 

Basic  marksmanship  knowledge 

1. 

Months  since  last  sustainment- 
level  training  on  marksmanship 

Medium 

performers 

1. 

Basic  marksmanship  knowledge 

1, 

Most  recent  record-fire  score 

2. 

Shot  group  depiction 

2. 

Whether  a  shooting  coach 

3. 

Self-regulation  (classroom 
training;  mean  of  worry  and 
planning  scales) 

3. 

Months  since  last  sustainment- 
level  training  on  marksmanship 

Low  performers 

1. 

Shot  group  depiction 

1. 

Months  since  last  sustainment- 
level  training  on  marksmanship 

2. 

Perceived  utility  of 
marksmanship  knowledge  to 
shooting  performance 

2. 

Number  of  hours  per  year 
shooting  as  part  of  USMC  duties 

3. 

Self-regulation  (classroom 
training;  worry  scale) 

# 
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For  learners,  basic  marksmanship  knowledge,  shot  group  depiction,  number  of 
years  in  service,  and  number  of  years  of  shooting  experience  prior  to  joining  USMC 
made  a  good  set  of  predictors.  A  participant  in  the  learner  group  tended  to  fire  more 
accurately  by  2.9  points  when  shot  group  knowledge  increased  by  1  standard 
deviation  (1.2  points),  holding  constant  all  the  other  variables  in  the  regression;  he  is 
also  expected  to  fire  more  accurately  by  1.6  and  1.4  points  when  the  number  of  years 
in  service  and  the  number  of  years  of  shooting  experience  prior  to  joining  USMC 
increase  by  1  standard  deviation  (1.5  and  4.8  years  respectively).  These  magnitudes 
of  relationships  are  based  on  the  leave-one-out  analysis,  given  the  fact  that  one 
person  cannot  be  considered  a  "learner"  from  a  substantive  perspective  if  he  has  a 
high  score  on  a  previous  occasion  and  a  low  score  on  a  later  occasion. 

Table  21 


Regression  Summary  for  Learners 


Variable 

Coefficient 

SE 

p  value 

Basic  marksmanship  knowledge 

0.23 

0.32 

.47 

Shot  group  depiction 

3.07 

1.41 

.05 

Number  of  years  in  service 

1.75 

1.01 

.11 

Number  of  years  of  shooting  experience  prior  to  joining  USMC 

Leave-one-out  analysis 

0.34 

0.26 

.21 

Basic  marksmanship  knowledge 

0.15 

0.21 

.49 

Shot  group  depiction 

2.45 

0.93 

.02 

Number  of  years  in  service 

1.07 

0.68 

.14 

Number  of  years  of  shooting  experience  prior  to  joining  USMC 

0.24 

0.17 

.19 

For  high  performers,  basic  marksmanship  knowledge  and  months  since  last 
sustainment-level  training  on  marksmanship  achieved  some  prediction,  although 
there  were  no  good  predictors  of  performance  for  high  performers.  Possible 
technical  reasons  are  the  restrictive  range  of  regression  outcome  (i.e.,  record-fire 
performance),  ranging  from  40  to  58,  and  thus  possibly  restrictive  range  of 
predictors,  and  the  small  sample  size  of  15.  One  substantive  reason  might  be  that, 
for  high  performers  who  are  well  past  the  cognitive  phase  of  skill  acquisition,  firing 
has  already  been  automatized  such  that  cognitive  knowledge  or  background  is  not 
predictive  of  the  performance  any  longer. 
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Table  22 

Regression  Summary  for  High  Performers 

Variable 

Coefficient 

SE 

p  value 

Months  since  last  sustainment-level  training  on  marksmanship 

-0.31 

0.26 

.26 

Basic  marksmanship  knowledge 

0.06 

0.28 

.84 

Leave-one-out  analysis 

Months  since  last  sustainment-level  training  on  marksmanship 

-0.34 

0.20 

.12 

Basic  marksmanship  knowledge 

0.23 

0.23 

.34 

For  medium  performers,  basic  marksmanship  knowledge,  shot  group 
depiction,  and  self-regulation  on  classroom  training,  among  the  cognitive  measures, 
and  most  recent  record-fire  score,  months  since  last  sustainment-level  training  on 
marksmanship,  and  shooting  coach  status,  among  the  background  measures, 
achieved  some  prediction.  This  set  of  covariates  is  very  similar  to  the  results  from 
analyzing  the  entire  sample,  in  fact,  sharing  the  same  set  of  background  measures 
and  a  subset  of  cognitive  measures.  This  result  can  be  thought  of  as  cross-validation, 
verifying  the  analysis  of  the  entire  sample.  A  participant  who  has  been  a  shooting 
coach  tended  to  score  5.8  points  higher  in  his  record-fire  performance,  holding 
constant  all  the  other  variables  in  the  regression  (p  <  .05).  Basic  marksmanship 
knowledge  and  self-regulation  reached  statistical  significance  between  .05  and  .10. 
Controlling  for  all  the  other  variables,  a  participant  who  is  a  medium  performer 
tended  to  score  1.5  points  higher  in  his  record-fire  performance  when  basic 
marksmanship  knowledge  or  self-regulation  on  classroom  training  were  higher  by  1 
standard  deviation,  5.5  and  0.6  points  respectively. 
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Table  23 


Regression  Summary  for  Medium  Performers 


Variable 

Coefficient 

SE 

p  value 

Most  recent  record-fire  score 

0.05 

0.16 

.74 

Basic  marksmanship  knowledge 

0.27 

0.16 

.09 

Shot  group  depiction 

0.20 

0.77 

.80 

Whether  took  coaches  course 

5.78 

2.81 

.04 

Months  since  last  sustainment-level  training  on 
marksmanship 

-0.04 

0.11 

.71 

Mean  of  self-regulation  planning  and  checking  scales 

2.69 

1.35 

.05 

For  low  performers,  shot  group  depiction,  perceived  value  of  knowledge  to 
shooting  performance,  and  self-regulation  on  classroom  training  (worry  scale) 
showed  up  among  cognitive  measures,  while  months  since  last  sustainment-level 
training  on  marksmanship  and  number  of  hours  per  years  shooting  as  part  of  USMC 
duties  did  among  background  measures.  Participants  who  were  low  performers 
tended  to  score  5.2  points  higher  in  their  record-fire  performance,  if  their  last 
sustainment-level  training  on  marksmanship  was  more  recent  by  6.7  months  (1 
standard  deviation),  holding  constant  all  other  variables  in  the  regression  (p  <  .05); 
they  also  tended  to  fire  more  accurately  by  4.4  points  as  the  perceived  value  of 
knowledge  to  shooting  performance  increased  by  1  standard  deviation  (1.1  points). 

Table  24 


Regression  Summary  for  Low  Performers 


Variable 

Coefficient 

SE 

p  value 

Mean  of  self-regulation  planning  and  checking  scales 

Perceived  utility  of  marksmanship  knowledge  to  shooting 

2.12 

1.60 

.21 

performance 

4.05 

2.06 

.08 

Months  since  last  sustainment-level  training  on  marksmanship 

-0.78 

0.33 

.04 

Number  of  hours  per  year  shooting  as  part  of  USMC  duties 

0.02 

0.05 

.64 

Self-regulation  worry 

-4.54 

4.42 

.33 
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To  what  extent  can  record-fire  performance  be  predicted  from  the  combinations  of 
perceptual-motor  (proxies),  cognitive,  and  affective  measures? 

As  with  the  entire  sample  analysis,  the  level  of  prediction  will  be  assessed  by 
two  criteria:  the  square  of  multiple  correlation  (R^)  and  the  95%  confidence  interval 
of  predicted  record-fire  performance.  Table  25  presents  multiple  correlations  (R)  and 
the  squares  of  multiple  correlations  (R^)  from  the  multiple  regression  analyses  of  the 
four  groups.  One  can  see  that  the  level  of  prediction  is  fairly  high  in  groups  of 
learners  and  low  performers.  Sixty-six  percent  of  the  total  variability  in  record-fire 
performance  is  accounted  for  in  low  performers,  while  62%  of  the  total  variability  is 
accounted  for  in  learners  (leave-one-out  analysis).  The  predictions  for  high 
performers  and  low  performers  were  moderate,  12%  and  22%  of  the  total  variability 
in  record-fire  performance  being  accounted  for  respectively.  This  suggests  that 
learners'  and  low  performers'  record-fire  scores  could  be  predicted  very  well  from 
the  set  of  covariates  shown  in  Table  21  and  Table  24,  respectively.  However,  the 
findings  from  subgroup  analyses  must  be  interpreted  with  caution  due  to  the  small 
sample  sizes  of  the  subgroups.  Note  that  only  15  to  17  participants  are  available  in 
groups  of  learners,  high  performers,  and  low  performers.  The  findings  are  sample- 
dependent  and  could  be  unstable. 

Table  25 


Multiple  Correlation  and  the  Square  of  Multiple  Correlation  of  Regression  by 
Subgroup 


Learner 

High  performer 

Entire 

Leave-one- 

Low 

Medium 

Entire 

Leave-one- 

subgroup 

out 

performer  performer 

subgroup 

out 

(n  =  17) 

(«  =  16) 

(n  =  17) 

(w  =  69) 

{n  =  15) 

(n  =  14) 

R  .76 

.79 

.81 

.47 

.35 

.56 

R2  .59 

.62 

.66 

.22 

.12 

.31 

The  95%  confidence  intervals  of  each  predicted  value  are  plotted  in  Figure  9  to 
Figure  12,  in  the  order  of  learners,  high  performers,  medium  performers,  and  low 
performers.  For  learners  and  high  performers,  confidence  intervals  from  the  leave- 
one-out  analyses  are  plotted. 
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The  prediction  achieved  more  accuracy  than  the  analysis  for  the  entire  sample, 
with  average  lengths  of  confidence  intervals  being  7.57,  8.07,  and  7.21  for  learners, 
high  performers,  and  medium  performers,  respectively  (the  average  length  of  the 
whole-group  analysis  was  around  10).  However,  the  lengths  of  confidence  intervals 
for  the  low  performers  tended  to  be  greater  than  those  of  the  whole-group  analysis. 
This  is  mainly  due  to  the  fact  that  the  predictors  chosen  in  the  analysis  of  low 
performers  had  small  ranges.  The  restricted  ranges  of  predictors  could  lead  to  larger 
errors  estimating  the  regression  coefficient  and  thus  to  larger  confidence  intervals. 


Figure  9.  The  95%  confidence  interval  of  predicted  values  from  analyses  of  learners. 
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Figure  21.  The  95%  confidence  interval  of  predicted  values  from 
analyses  of  medium  performers. 
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Figure  12.  The  95%  confidence  interval  of  predicted  values  from  analyses  of  low 
performers. 

Reliability  and  Validity  of  the  Measures  of  Rifle  Marksmanship  Knowledge 

How  reliable  are  our  measures  of  rifle  marksmanship  knowledge?  To  answer 
this  question,  we  calculated  coefficient  alphas  for  all  the  cognitive  and  affective 
measures  on  rifle  marksmanship.  Coefficient  alpha  is  also  referred  to  as  Cronbach's 
coefficient  alpha,  and  it  suggests  the  internal  consistency  of  a  test  by  estimating  the 
average  correlation  of  items  within  a  test.  When  multiple  items  in  a  test  are  designed 
to  measure  a  construct,  coefficient  alpha  could  suggest  the  reliability  of  the  test. 
Even  though  there  is  no  legitimate  cutoff  point,  a  coefficient  alpha  above  .70  would 
be  acceptable.  Table  26  presents  the  number  of  items  within  each  measure,  available 
participants  for  the  calculation,  and  the  coefficient  alphas. 
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Table  26 


Scale  Reliabilities  of  Cognitive  and  Affective  Measures 


Measures 

n 

Cronbach's 

alpha 

Number  of 
items 

Basic  marksmanship  knowledge 

161 

.83 

43 

Shot  group  depiction 

154 

.26 

5 

Self-regulation  planning 

156 

.86 

8 

Self-regulation  worry 

156 

.91 

8 

Self-regulation  checking 

156 

.83 

8 

Self-regulation  effort 

156 

.84 

8 

For  basic  marksmanship  knowledge,  the  coefficient  alpha  was  .84  with  43 
items.  The  total  score  of  the  43  items  was  used  for  analyses,  of  which  the  reliability 
was  ensured.  On  the  other  hand,  the  coefficient  alpha  for  shot  group  depiction 
knowledge  was  only  .27  with  5  items.  The  bivariate  correlations  among  the  5  items 
showed  that  the  items  were  independent  of  each  other  rather  than  interrelated. 
Among  the  10  pair-wise  correlations,  only  one  pair  turned  out  to  be  significant 
(flinching  and  bucking,  r  =  .20).  The  total  score  of  the  5  items  was  used  for  analyses, 
but  there  remains  a  question  if  this  scale  or  test  captures  a  well-defined  construct. 
The  four  subscales  of  self-regulation  planning,  worry,  checking,  and  effort  all 
showed  high  scale  reliability,  ranging  from  .83  to  .92.  For  the  other  cognitive 
measures  that  were  used  in  analyses,  coefficient  alpha  was  not  applicable.  Perceived 
utility  of  marksmanship  knowledge  to  shooting  performance  is  measured  by  1  item. 

What  is  the  quality  of  the  validity  evidence  for  our  measures  of  rifle 
marksmanship  knowledge?  In  order  to  address  the  validity  issue  of  our  measures 
of  rifle  marksmanship  knowledge,  the  measures  were  examined  from  various 
perspectives:  the  predictive  validity  and  the  incremental  validity  of  sets  of  cognitive 
measures  and  their  sensitivities  to  differentiating  groups  in  record-fire  performance. 

Predictive  validity.  To  see  the  quality  of  the  predictive  validity  of  cognitive 
and  affective  measures  on  rifle  marksmanship  knowledge,  first  we  examined 
bivariate  correlations  between  each  of  the  measures  and  record-fire  performance. 
Table  27  presents  the  results.  For  the  entire  sample,  all  knowledge  measures  other 
than  knowledge  map  showed  a  positive  and  significant  relationship  with  record-fire 
performance.  For  the  subsamples,  many  correlations  did  not  reach  statistical 
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significance  mainly  due  to  small  sample  sizes.  However,  one  can  still  see  that  the 
magnitudes  of  correlations  are  considerable  and  that  the  directions  are  as  expected. 
For  learners,  basic  marksmanship  knowledge  and  shot  group  depiction  were  highly 
and  significantly  correlated  with  record-fire  performance. 


Table  27 

Correlation  Between  Record-Fire  Performance  and  Cognitive  Measures  for  the  Entire  Sample  and  by 
Subgroup.  Sample  Size  Shown  in  Parentheses. 


BMK 

SG 

ESP 

KM 

SRP 

SRW 

PU 

Entire 

sample 

.29**  (156) 

.26**  (151) 

.19*  (155) 

.12  (148) 

.16  (153) 

.22**  (153) 

.19*  (154) 

Subgroup 

Learners 

.55*  (17) 

.52*  (17) 

- 

- 

- 

- 

~ 

High 

.29  (16) 

- 

- 

- 

- 

- 

- 

Medium 

.20  (98) 

.13  (94) 

- 

- 

.23*  (97) 

.24*  (97) 

- 

Low 

~ 

.36  (19) 

- 

- 

- 

-.20  (18) 

.36  (19) 

Note.  BMK  =  basic  marksmanship  knowledge.  SG  =  shot  group  depiction.  ESP  =  evaluation  of  shooter 
positions.  KM  =  knowledge  map.  SRP  =  self-regulation,  planning  scale.  SRW  =  self-regulation,  worry 
scale.  PU  =  perceived  utility  of  marksmanship  knowledge. 


Second,  we  examined  the  predictive  validity  of  our  cognitive  and  affective 
measures  as  a  set  instead  of  individually,  conducting  multiple  regression  analyses 
including  only  cognitive  and  affective  measures  as  predictors.  The  bivariate 
correlations  suggest  the  magnitudes  of  predictive  validity  of  individual  measures, 
whereas  the  multiple  correlation  or  the  square  of  the  multiple  correlation  in  the 
multiple  regressions  suggests  the  overall  predictive  validity  that  a  set  of  cognitive 
measures  simultaneously  have. 

Table  28  presents  the  results  of  all  multiple  regression  analyses  for  the  entire 
sample  and  for  the  subsamples.  The  set  of  cognitive  and  affective  measures  varied 
across  analyses;  sets  for  all  analyses  were  summarized  in  Table  20.  When  all 
participants  are  analyzed  together,  the  set  of  cognitive  and  affective  measures  (i.e., 
basic  marksmanship  knowledge,  shot  group  depiction,  proper  position 
identification,  knowledge  map,  self-regulation  with  respect  to  classroom  training, 
perceived  value  of  knowledge  to  shooting  performance)  accounted  for  17%  of  the 
total  variability  in  record-fire  scores.  For  learners,  40%  (46%  for  the  leave-one-out 
analysis)  were  accounted  for;  for  high  performers,  9%  (25%  for  the  leave-one-out 
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analysis);  for  medium  performers,  12%;  and  for  low  performers,  46%  were 
accounted  for  only  by  cognitive  and  affective  measures,  setting  aside  any 
background  measures.  Overall,  one  can  conclude  that  the  cognitive  measures 
contributed  to  prediction  to  a  considerable  extent. 

Table  28 

Multiple  Correlation  and  the  Square  of  Multiple  Correlation  of  Regression  on  Cognitive  Measures  by 
Subgroup 


Subsample  analysis 


Learner 


High  performer 


Entire 

Entire 

Leave-one- 

Low 

Medium 

Entire 

Leave-one- 

sample 

subgroup 

out 

performer  performer 

subgroup 

out 

(N  =  103) 

(«  =  17) 

(«  =  16) 

in  =  17) 

(«  =  69) 

[n  =  15) 

(n  =  14) 

R  .41 

.63 

.68 

.68 

.34 

.29 

.50 

R2  .17 

.40 

.46 

.46 

.12 

.09 

.25 

Incremental  validity.  Concerning  the  validity  evidence  for  our  cognitive  and 
affective  measures,  one  important  aspect  is  the  incremental  validity.  Even  though 
the  cognitive  and  affective  measures  are  predictive  of  the  record-fire  performance,  if 
the  measures  provide  only  overlapped  prediction  with  background  measures,  then 
the  usefulness  of  our  measures  will  substantially  decrease  given  that  the 
background  measures  are  much  easier  to  collect.  Incremental  validity  here  asks  the 
question  if  the  cognitive  and  affective  measures  achieve  additional  prediction  of 
record-fire  performance  over  and  beyond  the  background  measures.  Table  29 
presents  incremental  validity  for  the  entire  sample  and  for  subsamples  in  two  scales: 
the  multiple  correlation  (R)  and  the  square  of  the  multiple  correlation  {R^). 
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Table  29 

Incremental  Validity  (in  Percent)  of  Cognitive  Measures  With  Respect  to  Background  Variables 


Subsample  analysis 


Learner  High  performer 


Entire 

Entire 

Leave-one- 

Low 

Medium 

Entire 

Leave-one- 

Incremental 

Validity 

sample 

subgroup 

out 

performer 

performer 

subgroup 

out 

Measure 

{N  =  103) 

(«  =  17) 

(«  =  16) 

{«  =  17) 

(«  =  69) 

{n  =  15) 

(«  =  14) 

AR(%) 

7.25 

31.35 

40.99 

9.99 

11.69 

0.44 

5.88 

ARH%) 

7.61 

38.42 

48.12 

15.35 

9.66 

0.32 

6.25 

Results  of  the  multiple  regression  analyses  indicated  that  the  measures  of 
knowledge  of  rifle  marksmanship  achieved  incremental  validity  over  background 
measures,  record-fire  performance  being  the  criterion.  For  the  entire  sample, 
increased  by  around  8%  due  to  cognitive  measures  after  accounting  for  background 
measures.  For  the  subsamples,  the  incremental  validity  is  remarkably  high  in 
learners  (R^  increase  =  38%  and  48%  for  the  leave-one-out  analysis),  fairly 
considerable  in  medium  performers  and  low  performers  (R2  increase  =  10%  and  15% 
respectively),  and  the  least  but  still  significant  in  high  performers  with  the  outlier 
eliminated  (R^  increase  =  6%;  0.3%  with  the  outlier). 

Sensitivity  to  group  differences.  One  goal  regarding  prediction  of  record-fire 
performance  is  to  distinguish  low  performers  from  others,  so  that  one  can  keep  from 
attempting  to  qualify  Marines  who  are  not  able  to  fire  as  accurately  as  expected.  In 
relation  to  this,  one  aspect  of  validity  evidence  for  cognitive  measures  would  be  to 
show  differences  between  low  performers  and  the  others.  To  address  this  question, 
we  conducted  t  tests  of  cognitive  measures  comparing  low  performers  and  others. 
The  selection  of  low  performers  is  based  on  the  classification  in  Figure  8.  Results  of  t 
tests  showed  that  low  performers  (M  =  24.20,  SD  =  3.85),  compared  to  others  (M  = 
27.26,  SD  =  5.49),  scored  significantly  lower  on  the  basic  marksmanship  measure  ip  = 
.02).  There  were  no  other  significant  differences  on  any  of  the  other  knowledge 
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Summary  and  Discussion  of  Study  1 

Reliability  of  measures.  Overall,  our  measures  demonstrated  moderate  to  high 
reliabilities.  The  lowest  reliability  was  with  the  shot  group  depiction  task  (a  =  .26). 
The  basic  marksmanship  knowledge  measure  had  high  a  (.83).  The  survey  measures 
(i.e.,  affective)  demonstrated  high  reliabilities  too,  with  a  in  the  .8  to  .9  range. 

Predictive  validity.  Overall,  the  best  predictor  of  record-fire  score  was  most 
recent  record-fire  score  (r  =  .41,  p  <  .01).  Basic  marksmanship  knowledge,  proper 
position  identification,  and  number  of  months  since  the  last  Phase  I  training  were 
related  to  record-fire  score  in  the  .2  to  .4  range  at  the  .10  significance  level.  When 
subgroup  analyses  were  conducted  (forming  groups  based  on  mean  scores  and 
standard  deviations  using  record-fire  scores  across  occasions),  multiple  regression 
analyses  yielded  Rs  from  .4  to  .8  across  the  different  subgroups. 

Construct  validity.  The  subgroup  analyses  provided  preliminary  evidence  that 
different  variables  mattered  for  different  shooters.  The  subgroup  classification  was 
based  on  the  expectation  that  the  variability  in  record-fire  scores  would  reflect 
where  a  participant  was  with  respect  to  skill  development  (i.e.,  more  varied 
performance  would  indicate  a  shooter  still  in  the  cognitive  phase,  while  more  stable 
performance  would  indicate  a  shooter  further  along  in  development). 

Incremental  validity  analyses  showed  substantial  contribution  of  knowledge 
measures  to  the  prediction  of  record-fire  score.  Overall,  across  the  entire  sample,  the 
set  of  cognitive  and  affective  measures  accounted  for  17%  of  the  total  variability  in 
record-fire  scores.  For  learners  only,  cognitive  and  affective  variables  accounted  for 
40%  of  the  variance,  9%  for  high  performers,  12%  for  medium  performers,  and  46% 
for  low  performers.  These  values  are  above  and  beyond  the  variance  accounted  for 
by  background  measures.  However,  these  results  should  be  taken  as  preliminary  as 
the  sample  sizes  for  the  subgroups  were  small. 

Overall,  the  results  of  Study  1  show  a  potentially  strong  cognitive  component 
to  shooting  performance.  Subgroup  analyses  suggested  that  different  sets  of 
variables  mattered  for  participants  classified  as  learners,  low  performers,  medium 
performers,  and  high  performers.  In  general,  background  and  knowledge  measures 
contributed  to  the  prediction  of  record-fire  scores  for  learners,  low  performers,  and 
medium  performers.  There  were  no  good  predictors  for  high  performers. 
Incremental  validity  analyses  showed  that  knowledge  measures  contributed 
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substantially  to  the  prediction  of  record-fire  scores,  above  and  beyond  background 
measures  alone. 

Study  2— Method 

The  purpose  of  Study  2  was  to  replicate  our  study  at  Quantico,  VA.  We  also 
refined  existing  measures  and  tested  new  ones,  given  the  results  of  Study  1.  The 
reasons  behind  the  replication  effort  were  (a)  potential  sample  difference,  and  (b) 
differences  in  the  quality  of  the  marksmanship  instruction  (i.e.,  for  Phase  I,  Quantico 
uses  trained  instructors). 

Participants 

One-hundred  fifty-two  sustainment-level  Marines  participated  in  the  study  at 
WTBN  Quantico,  VA.  A  description  of  the  sample  is  shown  in  Table  30  through 
Table  33.  In  general,  the  participants  were  male  (94%),  enlisted,  with  less  than  4 
years  in  service,  and  from  a  mix  of  support,  base,  and  combat  units.  The  mean  prior 
qualification  scores  ranged  from  mid-  to  high  sharpshooter.  In  addition,  a  few 
participants  reported  completing  various  coaches  courses.  Compared  to  the  sample 
in  Study  1,  the  current  sample  appeared  older  and  slightly  more  experienced  in 
shooting. 

Table  30 


Descriptive  Statistics  on  Background  Variables 


Variable 

n 

M 

SD 

Min. 

Max. 

Most  recent  qualification  score® 

139 

37.2 

9.6 

0 

62 

Second  most  recent  qualification  score® 

89 

38.9 

8.7 

23 

65 

Third  most  recent  qualification  score® 

58 

39.2 

9.6 

23 

65 

Age 

143 

23.2 

4.0 

18 

38 

Years  in  service 

140 

3.8 

3.4 

0.75 

19 

Frequency  of  shooting  as  part  of  duties'’ 

145 

2.0 

1.2 

1 

5 

Frequency  of  shooting  outside  of  duties'® 

145 

2.5 

1.4 

1 

5 

Years  of  shooting  experience  before  Marines 

142 

4.0 

5.2 

0 

16 

®0  -  24  =  unqualified;  25  -  34  =  marksman;  35 

-  39  =  sharpshooter;  40 

-65  = 

expert.  '’1  = 

Never;  2  = 

Once  or  twice;  3  =  A  few  times;  4  =  Often;  5  =  Very  often. 


70 


CSE  Deliverable 


Table  31 


Distribution  of  Participants  by  Combat  Status 


Frequency 

Percent 

Division** 

40 

30.8 

FSSG* 

14 

10.8 

Support/ Base*^ 

60 

39.5 

Other*^ 

16 

12.3 

*>Combat  arms  and  combat  support.  ^Force  Service 
Support  Group.  ‘^Base  and  formal  schools. 
^Aviation. 

Table  32 

Distribution  of  Participants  by  Rank 

Rank 

Frequency 

Percent 

Private  (El) 

1 

0.6 

Private  First  Class  (E2) 

8 

5.0 

Lance  Corporal  (E3) 

60 

37.5 

Corporal  (E4) 

43 

26.9 

Sergeant (E5) 

32 

20.0 

Staff  Sergeant  (E6) 

12 

7.5 

Master  Sergeant  (E8) 

1 

0.6 

Captain  (03) 

3 

1.9 

Table  33 


Distribution  of  Participants  by  Marksmanship  Instructor  Training 


Course 

No.  of  Participants  Who 

Completed  Course 

Marksmanship  Coaches  Course  (MOS  8530) 

3 

Marksmanship  Instructor  (MOS  8531) 

5 

Small  Arms  Weapons  Instructor  (MOS  8532) 

3 

Range  Officer  (MOS  9925) 

1 
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Participants  were  given  up  to  four  days  of  live-fire  practice.  A  fifth  day  was  for 
qualification.  However,  anyone  could  elect  to  attempt  to  qualify  on  the  third  or 
fourth  day  of  live-fire  practice.  All  remaining  participants  were  required  to  attempt 
to  qualify  on  qualification  day. 

During  practice,  participants  could  get  help  from  range  coaches.  During  a 
qualification  attempt,  participants  did  not  receive  any  help.  If  participants  failed 
their  first  qualification  attempt,  they  were  provided  with  additional  coaching  and 
given  the  opportunity  to  attempt  to  qualify  again  on  subsequent  days.  Marines  who 
failed  the  second  qualification  attempt  were  at  risk  of  being  dropped  from  the 
Marine  Corps. 

Design 

The  tasks  were  administered  to  six  groups  over  eight  days.  The  first  group  was 
a  dedicated  group  tested  five  out  of  the  eight  days.  The  remaining  groups  each 
contained  different  participants.  This  design  reflected  availability  of  participants, 
limitations  in  classroom  testing  space,  and  limitations  in  the  number  of  computers. 
There  were  many  more  available  participants  than  could  be  accommodated  in  a 
single  setting. 

Measures 

The  following  measures  were  provided  to  us  by  the  USMC  or  administered  to 
Marines: 

Record-fire  score.  The  main  outcome  measure  was  the  Marine's  record-fire 
score.  This  score  was  the  official  score  on  record  from  the  USMC  database. 

Most  recent  record-fire  scores.  These  were  self-reported  qualification  scores  for 
the  three  most  recent  qualification  trials. 

Knowledge  mapping.  After  Study  1,  coaches  and  staff  from  WTBN  Quantico 
reviewed  the  knowledge  map  task  and  revised  the  sets  of  terms  and  links.  The 
revised  set  of  terms  were  3  elements  of  a  good  shooting  position,  7  factors  common  to  all 
shooting  positions,  aiming  process,  bone  support,  breath  control,  consistency,  controlled 
muscular  tension,  eye  on  front  sight  post,  eye  relief,  finger  placement,  follow-through, 
forward  elbow  placement,  fundamentals  of  marksmanship,  grip  of  firing  hand,  muscular 
relaxation,  natural  point  of  aim,  natural  respiratory  pause,  placement  of  buttstock  in 
shoulder,  rear  elbow  placement,  sight  alignment,  sight  picture,  stable  firing  position,  stock 
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weld  placement,  and  trigger  control.  The  set  of  links  were  affects,  decreases,  follows, 
happens  during,  helps,  increases,  leads  to,  part  of,  requires,  and  uses.  The  format  of  this 
measure  was  online. 

Scoring  was  done  using  the  same  method  as  Study  1,  except  we  used  three 
criterion  maps  to  score  student  maps  against.  The  criterion  maps  were  generated  by 
three  subject  matter  experts.  These  were  primary  marksmanship  instructors  at 
Quantico  whose  job  was  to  teach  rifle  marksmanship.  A  participant's  score  was  the 
total  number  of  propositions,  across  the  three  criterion  maps,  in  his  or  her  map  that 
were  also  in  the  criterion  maps.  The  criterion  map  is  given  in  Appendix  J  (Expert  1, 
7,  8,  screen  shots)  and  Appendix  K  (Expert  1,  7,  8,  propositions). 

Basic  marksmanship  knowledge.  We  used  a  revised  measure  based  on  Study 
1.  Changes  were  made  as  a  result  of  review  by  Quantico  WTBN  coaches  and  staff.  In 
addition,  we  added  about  6  items  that  required  participants  to  consider  hypothetical 
situations  and  8  items  that  asked  participants  to  predict  effects  on  the  weapon  given 
a  range  of  movement  on  the  part  of  the  shooter.  The  purpose  of  these  items  was  to 
test  for  knowledge  of  causal  relations  in  a  selected-response  format.  The  format  of 
this  measure  was  paper.  A  copy  of  the  measure  is  given  in  Appendix  L. 

Shot  group  depiction.  We  used  a  paper  version  of  the  measure  from  Study  1. 
The  format  of  this  measure  was  paper.  A  copy  of  the  measure  is  given  in  Appendix 
M. 

State  anxiety.  We  adopted  a  measure  from  O'Neil  and  Herl  (1998)  to  measure 
participants'  state  anxiety  about  qualification  prior  to  and  after  qualification.  The 
format  of  this  measure  was  paper. 

Trait  and  state  worry.  We  adopted  a  measure  from  O'Neil  and  Herl  (1998)  to 
measure  participants'  worry  about  qualification  in  general  (trait  measure)  and  just 
before  or  just  after  qualification  (state  measure).  The  format  of  these  measures  was 
paper.  The  trait  measure  is  given  in  Appendix  N  and  the  state  anxiety  and  state 
worry  items  are  given  in  Appendix  O. 

Firing  line  experience.  This  was  a  new  measure  that  we  developed  to  gather 
information  on  participants'  overall  experience  on  the  firing  line.  The  format  of  this 
measure  was  paper.  In  addition,  we  administered  a  subset  of  the  items  to 
participants  after  they  had  qualified.  A  copy  of  the  measure  is  given  in  Appendix  P. 
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Perceived  level  of  marksmanship  knowledge.  This  was  a  new  measure  that 
we  developed  to  gather  participants'  self-reported  knowledge  of  the  fundamentals. 
The  format  was  paper.  A  copy  of  the  measure  is  given  in  Appendix  Q. 

Perceived  utility  of  marksmanship  knowledge.  This  was  a  new  measure  that 
we  developed  to  gather  participants'  perceptions  of  the  utility  of  knowing  the 
fundamentals  of  rifle  marksmanship.  The  format  was  paper.  A  copy  of  the  measure 
is  given  in  Appendix  R. 

Background  survey.  We  used  the  same  measure  from  Study  1  with  slight 
modifications  to  wording.  We  also  asked  Marines  how  much  they  know  about  rifle 
marksmanship,  their  perceived  importance  of  knowledge  of  marksmanship  to 
shooting  performance,  and  the  difficulty  of  our  assessments.  The  format  of  this 
measure  was  paper.  A  copy  of  the  measure  is  given  in  Appendix  S. 

Evaluation  of  shooter  positions.  We  used  the  same  measure  from  Study  1.  The 
format  of  this  measure  was  online. 

Reliability  of  measures.  Table  34  shows  the  reliabilities  of  the  measures.  In 
general,  the  measures  showed  high  reliabilities.  As  in  Study  1,  the  shot  group 
measure  had  very  low  reliabilities.  The  basic  knowledge  measure  showed  a  decrease 
in  reliability  over  Study  1.  While  we  had  made  some  changes  to  the  measures,  the 
changes  were  refinements.  Reliabilities  for  the  other  measures  were  acceptable. 
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Table  34 


Reliability  of  Measures 


n 

Number 

Cronbach's 

Measure 

of  items 

alpha 

Knowledge  map 

144 

3 

.92 

Basic  marksmanship  knowledge 

113 

37 

.73 

Evaluation  of  shooter  positions 

141 

6 

.64 

Shot  group  depiction 

144 

5 

.31 

Trait  worry 

139 

8 

.93 

State  anxiety 

119 

5 

.85 

State  anxiety  (post-qual.) 

51 

5 

.89 

State  worry 

118 

6 

.79 

State  worry  (post-qual.) 

51 

6 

.69 

Firing  line  experience 

143 

6 

.83 

Firing  line  experience  (post-qual.) 

33 

6 

.58 

Perceived  level  of  marksmanship 
knowledge 

143 

5 

.95 

Perceived  utility  of  marksmanship 
knowledge 

141 

6 

.79 

Procedure 

Measures  were  administered  to  participants  in  the  order  and  allotted  times 
shown  in  Table  35,  across  a  total  of  eight  occasions  as  shown  in  Table  36. 
Participants  completed  the  tasks  after  they  had  completed  classroom  training  or  live 
fire  for  that  day.  In  general,  participants  completed  the  tasks  well  within  the  times 
listed  and  were  given  more  time  if  needed. 
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Table  35 

Tasks  and  Time  Allotted 

Time 

Task  allotted 

Introduction  to  study  5 

Knowledge  mapping  training  5 

Knowledge  mapping  task  20 

Evaluation  of  shooter  positions  training  5 

Evaluation  of  shooter  positions  task  20 

Background  survey  10 

Basic  marksmanship  knowledge  task  20 

Shot  group  depiction  task  instructions  and  task  10 

State  worry  survey  5 

Trait  worry  survey  5 

State  firing  line  experience  5 


Table  36 

Administration  Schedule 


Classroom  training 
(Phase  I) 

Live-fire  practice  and  qualification 
(Phase  II) 

Group 

n 

Pre 

Mid  Post 

Practice 

-only 

Practice  or 
attempt  to  qualify 

Qual¬ 

ification 

Postqual¬ 

ification 

1 

38 

ALL 

KM  KM 

KM 

ALL 

2 

22 

ALL 

3 

29 

ALL 

4 

20 

ALL 

5 

19 

ALL 

6 

15 

ALL 

Note.  ALL  =  indicates  administration  of  all  measures.  KM  =  Only  knowledge  map  measure  was 
administered. 
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Study  2— Results 


Preliminary  Analyses 

Prior  to  conducting  the  main  analyses,  we  checked  for  group  differences  on  the 
record-fire  scores  and  on  the  knowledge  measures.  Separate  one-way  ANOVAs 
were  conducted  to  check  for  differences  across  groups  on  (a)  record-fire  scores,  (b) 
knowledge  mapping  scores,  (c)  basic  marksmanship  knowledge  scores,  (d)  shot 
group  depiction  scores,  and  (e)  evaluation  of  shooter  positions  scores.  There  were  no 
significant  differences  on  any  of  the  measures  and  thus  we  pooled  the  data  across 
the  groups. 

Main  Analyses 

Table  37  to  Table  39  show  descriptive  statistics  of  the  perceptual-motor, 
cognitive,  and  affective  variables.  Table  40  to  Table  42  present  intercorrelations  and 
correlations  among  the  three  groups  of  variables.  The  mean  record-fire  score  is 
consistent  with  our  prior  studies,  suggesting  comparable  shooting  performance 
between  Stone  Bay  and  Quantico  Marines  undergoing  sustainment-level 
qualification.  In  addition,  participants  reported  generally  positive  firing  line 
experiences  in  the  past. 

Table  37 


Descriptive  Statistics  of  Perceptual-Motor  Variables 


Variable 

ft 

M 

SD 

Min. 

Max. 

Record-fire  score‘' 

138 

38.89 

8.25 

14 

56 

Most  recent  record-fire  scored 

138 

37.46 

9.09 

14 

62 

Second  most  recent  record-fire  scores 

89 

38.89 

8.71 

23 

65 

Third  most  recent  record-fire  score® 

58 

39.24 

9.64 

23 

65 

Frequency  of  shooting  outside  job^ 

145 

2.47 

1.43 

1 

5 

Years  of  shooting  experience  before  joining  Marines 

142 

4.05 

5.24 

0 

16 

Firing  line  experience‘s 

143 

2.92 

.60 

1.17 

4.00 

Firing  line  experience  (post-qual.)‘s 

33 

3.32 

.39 

2.50 

4.00 

^0  -  24  =  unqualified;  25  -  34  =  marksman,  35  -  39  =  sharpshooter;  40  -  65  =  expert.  '^1  =  Never,  2  = 
Once  or  twice,  3=  A  few  times,  4  =  Often,  5  =  Veiy  often.  <^1  =  Almost  never,  2  =  Sometimes,  3  =  Often,  4  = 
Almost  always  (higher  values  indicate  higher  positive  experience). 
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With  respect  to  marksmanship  knowledge  {Table  38),  participants  in  general 
reported  they  had  a  moderate  to  large  amount  of  knowledge  of  the  fundamentals  of 
marksmanship,  and  they  perceived  the  value  of  knowledge  of  the  fundamentals  to 
shooting  well.  Further,  when  tested  with  our  knowledge  measures,  performance  on 
the  basic  knowledge  measure  was  moderately  high  (mean  score  was  75%  correct). 
However,  on  the  more  complex  tasks  (i.e.,  knowledge  mapping,  shot  group 
depiction,  and  evaluation  of  shooter  positions),  overall  performance  was  low. 


Table  38 


Descriptive  Statistics  of  Knowledge  Variables 


Variable 

n 

M 

SD 

Min. 

Max. 

Basic  marksmanship  knowledge^ 

113 

27.22 

4.28 

9 

35 

Shot  group  depiction'’ 

144 

1.97 

1.15 

0 

4 

Evaluation  of  shooter  positions'’ 

141 

16.72 

6.18 

0 

28 

Knowledge  map 

144 

3.43 

5.11 

0 

28 

Perceived  level  of  marksmanship  knowledge'* 

143 

3.26 

0.65 

1 

4 

Perceived  utility  of  marksmanship  knowledge® 

144 

1.65 

0.44 

1 

3 

^Maximum  possible  is  37.  ■’Maximum  possible  is  5.  ‘’Maximum  possible  is  63.  ‘■1  =  Not  at  all,  2  =Some,  3 
=  Moderate  amount,  4  =  Very  much  (higher  values  indicate  more  self-reported  knowledge).  =  Strongly 
agree,  2  =  Agree,  3  =Disagree,  4  =  Strongly  disagree  (higher  values  indicate  lower  perceived  value  of 
marksmanship  knowledge). 

With  respect  to  affective  measures  (Table  39),  participants  in  general  reported 
low  (trait)  worry  about  qualification  in  general.  Similarly,  when  participants  were 
asked  to  estimate  how  worried  and  anxious  they  were,  participants  reported  low 
(state)  worry  and  anxiety.  Because  this  measure  was  administered  to  participants 
who  were  measured  on  different  days,  the  measure  asked  these  participants  to 
estimate  how  they  felt  during  qualification  (if  they  had  already  attempted  to  qualify) 
or  predict  how  they  would  feel  during  qualification  (if  they  were  going  to  attempt  to 
qualify  in  the  future).  For  the  group  of  participants  that  were  administered  the  state 
worry  and  anxiety  measures  after  they  had  qualified,  participants  in  this  group 
reported  a  similar  level  of  worry  and  anxiety  during  qualification. 
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Table  39 


Descriptive  Statistics  of  Affective  Variables 


Variable 

n 

M 

SB 

Min. 

Max. 

Trait  worry® 

139 

1.77 

0.75 

1 

4 

State  worry  (pre-qual.)'^ 

118 

1.62 

0.62 

1 

4 

State  worry  (post-qual.)*’ 

51 

1.75 

0.63 

1 

3.7 

State  anxiety  (pre-qual.)*’ 

119 

1.94 

0.68 

1 

4 

State  anxiety  (post-qual.)*’ 

51 

1.76 

0.77 

1 

4 

=  Almost  never,  2  =  Sometimes,  3  =  Often,  4  =  Almost  always  (higher  values 
indicate  higher  worry).  '’1  =  Not  at  all,  2  =Sometimes,  3  =  Moderately  so,  4  = 
Very  much  so  (higher  values  indicate  higher  worry/ anxiety). 


Bivariate  correlations  among  the  perceptual-motor  variables  (see  Table  40) 
indicate  low  to  moderate  relationships.  Especially  interesting  are  the  correlations 
with  the  record-fire  score.  The  best  predictor  of  record-fire  score  is  firing  line 
experience  administered  after  the  participants  qualified.  This  is  not  surprising  as  the 
participants  presumably  have  reasonable  recollection  of  their  overall  shooting 
experience.  Given  this  assumption,  the  magnitude  of  the  firing  line  measure  can  be 
thought  as  of  an  upper  bound  against  which  other  measures  can  be  compared. 

For  the  purposes  of  predicting  a  participant's  score  prior  to  qualification,  the 
most  recent  record-fire  score  correlates  with  record-fire  score  in  the  .3  -  .4  range, 
consistent  with  Pilot  Study  1  (r  =  .38,  p  <  .05),  Pilot  Study  2  (r  =  .36,  p  <  .01),  Study  1 
{r  =  .41,  p  <  .01),  and  prior  research  (e.g.,  Schendel  et  al.,  1983;  Smith  &  Hagman, 
2000,  Experiment  2). 
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Table  40 

Intercorrelations  Among  Perceptual-Motor  Variables 

Variable 

1 

2 

3 

4 

5 

6  7 

1.  Record-fire  score 

~ 

2.  Most  recent  record-fire  score 

.34** 

~ 

3.  Second  most  recent  record-fire  score 

.31* 

.37** 

~ 

4.  Third  most  recent  record-fire  score 

.27 

.56** 

.57** 

~ 

5.  Frequency  of  shooting  outside  job 

.27* 

.32** 

.18 

.46** 

~ 

6.  Years  of  shooting  experience  before 

.26* 

.40** 

.29* 

.37* 

7Q** 

- 

joining  Marines 

7.  Firing  line  experience  (pre-qual.) 

.33** 

41** 

.33* 

.43** 

.26* 

.30** 

8.  Firing  line  experience  (post-qual.) 

.57** 

.41* 

.45 

.61* 

.34 

.03  .74** 

Bivariate  correlations  between  the  perceptual-motor  variables  and  affective 
variables  (see  Table  41)  indicate  low  to  moderate  negative  relationships,  a  pattern 
consistent  with  prior  research  (e.g.,  Tierney  et  al.,  1979).  While  the  trait  worry 
measures  correlated  significantly  with  record-fire  score,  the  state  worry  measure  did 
not.  Comparing  the  state  worry  and  state  anxiety  measures  administered  before  the 
qualification  attempt  to  the  measure  administered  after  the  qualification  attempt 
suggests  interesting  findings.  First,  assuming  a  relationship  between  shooting 
performance  and  worry  and  anxiety  exists— it  appears  that  participants  who  were 
asked  to  predict  how  they  would  feel  at  qualification  (i.e.,  those  who  had  yet  to 
attempt  qualification)  were  poor  judges  of  their  situation.  Additional  evidence  for 
this  interpretation  is  seen  in  significant  and  moderate  correlations  between  pre¬ 
qualification  worry  and  anxiety  measures  and  the  pre-qualification  firing  line 
experience  measure,  but  non-significant  correlations  with  the  post-qualification 
firing  line  experience  measure.  In  contrast,  those  shooters  who  were  asked  to  reflect 
on  their  qualification  experience  were  much  better  judges,  as  indicated  by  the  higher 
relationship  between  the  state  worry  and  anxiety  measures  and  record-fire 
performance. 


Table  41 

Correlations  Between  Perceptual-Motor  Variables  and  Affective  Variables,  and  Intercorrelations  Among  Affective  Variables 


i 


0) 

Xi 

cd 

u 

O) 

> 

'S 

O 

U 


c 

o 


0) 

b 

o 

u 


fTi 


♦SS 

*u 

> 

u 

O 

4-» 

o 

I 

s 

a 

01 

u 

IH 

01 

CU 


o 

oi 

<n 


O 

u 

01 

C!S 


I 

tu 

hJ 

(Jh 


W 

1-1 

pH 


CP 

X 

PU 


O 

cn 

Ph 


■E  O  s 

^  E  « 


•d  «  c 

C  o  « 

<N  g  g 


«  c 
o  ^ 


c 

01 

fc 

3 

u 


X 

fd 

c3 

> 


•K 

4f 

Ln 

\o 


00 

fS 


(N 


* 

CO 

CO 


CO 


ON 

CN 


* 

m 

IT) 


04 


4c 


04 


O 


00 

o 


m 


o 

04 


O 

m 


a^ 

CO 


04 


00 

LO 


4c 

\£> 

m 


VO 


04 


O 

CO 


CO 


04  bx 


4c 

in 


04 


4c 


4c 

04 

VO 


IX 

04 


04 


04 


4c 

4c 

vO 


O 


04 


CO 

CO 


4c  4c 

VO  r-* 

04 


x—v 

• 

(d 

cd 

fd 

d 

fd 

d 

d 

cr* 

d 

cr 

a" 

1 

cr 

1 

<14 

1 

if) 

i 

if) 

4h 

o 

Ui 

o 

vSt 

3 

& 

to 

}h 

b 

b 

4>> 

.2 

-b 

u 

O 

o 

b> 

}-l 

o 

b> 

’x 

c 

‘x 

c 

> 

> 

> 

(d 

(d 

X 

2 

04 

04 

04 

cd 

2 

Id 

s 

"Jd 

H 

cn 

55 

55 

55 

r-t 

oi 

CO 

in 

O) 

u 

C 

.a 

‘C 

(D 

Oh 

X 

0) 

04 

g 

00 

C 


w 

X 

Uh 

<14 

.5 

‘C 

n3 

S 

t>0 

C 

.'s 

o 


(D 

X 

04 

U 

c 

‘C 

0) 

a 

X 

04 

00 

.£ 

‘X 

o 

o 

X 

if) 

UiM 

o 

Ul 

td 

04 

II 

Dh 

X 

w 

'n  ^ 

.2.  5 

mh  CT" 

o  z 
O)  g 

:2  a 

<45 

d  ^ 

o  ^ 
60  2 
C  ’E 
■■E  " 
O  CP 
O  X 
x  'IJ 

c/1  01 

>1  ,s 

CJ  Ph 

E  60 
01 

3  -c 
.b 
S 

pp  II 
II  tN 

0'S 

CD  rZ 

Ph  " 

v§  d 
<  (T 


i 


i 


qual).  FLE-2  =  Firing  line  experience  (post-qual.). 
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Bivariate  correlations  between  the  perceptual-motor  variables  and  cognitive 
variables  (see  Table  42)  indicate  virtually  no  relationship.  The  only  interesting 
relationship  found  was  with  participants'  self-reported  level  of  knowledge  of  the 
fundamentals  of  marksmanship.  This  was  the  only  knowledge-related  measure  that 
related  to  record-fire  score.  When  the  intercorrelations  among  the  knowledge 
measures  were  examined,  basic  knowledge  of  marksmanship  was  significantly 
related  to  other  measures  of  knowledge,  suggesting  some  overlap  in  the  constructs 
measured. 

Prediction  of  Record-Fire  Performance 

To  predict  record-fire  performance,  we  conducted  a  series  of  multiple 
regression  analyses  as  in  Study  1.  We  were  interested  in  replicating  the  whole-group 
and  subgroup  analyses;  however,  the  sample  size  and  distribution  precluded  the 
subgroup  analyses.  As  shown  in  Table  43  and  Figure  13,  there  were  low  numbers  of 
participants  classified  as  learners,  low  performers,  and  high  performers. 

A  comparison  with  Study  1  shows  similar  means  for  the  different  groups  with 
the  exception  of  the  learner  groups  (Study  1,  M  =  47.2;  see  Table  19).  Thus,  the 
regression  analyses  were  conducted  for  the  entire  group;  subgroup  analyses  based 
on  prior  record-fire  scores  were  not  performed. 

Table  43 


Descriptive  Statistics  of  Record-Fire  Performance  by  Subgroup 


n 

M 

SD 

Min. 

Max. 

Entire  sample 

138 

38.89 

8.25 

14 

56 

Learners 

10 

40.50 

8.87 

25 

51 

High  performers 

14 

48.14 

5.70 

35 

56 

Medium  performers 

88 

39.17 

6.17 

25 

51 

Low  performers 

13 

26.92 

8.63 

14 

45 
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Figure  13.  Subgroup  classification  based  on  scatterplot  of  average  outcome  scores 
(n  =  136). 


What  combinations  of  perceptual-motor  (proxies),  cognitive,  and  affective 
measures  yield  the  best  (practical)  predictors  of  record-fire  performance? 

As  in  Study  1,  multiple  regression  analyses  were  used  to  predict  record-fire 
scores.  The  dependent  variable  was  record-fire  score,  and  the  independent  variables 
were  chosen  based  on  theoretical  considerations  first,  then  empirical  considerations 
second.  That  is,  our  basic  model  included  perceptual-motor,  cognitive,  and  affective 
variables  and  the  particular  variables  chosen  within  each  category  were  based  on  the 
magnitude  of  the  bivariate  correlation  with  record-fire  score,  and  the  magnitude  of 
multiple  correlation  (R)  of  different  analyses.  In  addition,  the  number  of  variables  to 
include  in  the  model  was  limited  by  the  sample  size.  We  attempted  to  maintain  a 
ratio  of  number  of  cases  to  number  of  variables  to  at  least  20.  The  final  multiple 
regression  model  is  made  up  of  four  variables  shown  in  Table  44,  R  =  .52  (R^  =  27, 
adjusted  R^  =  .24),  SE  =  7.17.  A  plot  of  the  confidence  intervals  for  the  mean 
predicted  score  is  shown  in  Figure  14. 
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Table  44 


Multiple  Regression  Summary  for  Predicting  Record-Fire  Performance  (n  =  94) 


Variable 

Unstandardized  coefficient 

SE 

p  value 

Most  recent  record-fire  score 

.17 

.10 

.099 

Firing  line  experience  (pre-qual.) 

4.69 

1.44 

.002 

Basic  marksmanship  knowledge 

-.02 

.20 

.909 

Trait  worry 

-1.44 

1.22 

.241 

Individual  Participant 

Figure  14.  The  95%  confidence  interval  for  the  mean  predicted  record-fire  scores  (n  =  107). 

Incremental  validity.  An  incremental  validity  analysis  was  conducted  using 
hierarchical  regression  analyses.  The  purpose  of  this  analysis  was  to  examine  how 
much  additional  variance  the  cognitive  and  affective  measures  accounted  for  in  the 
prediction  of  record-fire  scores.  Table  45  shows  the  incremental  validity  of 
additional  variables  beyond  the  most  recent  record-fire  score.  The  order  of  the  entry 
was  based  on  the  presumed  order  of  importance  given  our  theoretical  framework. 
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The  addition  of  firing  line  experience  resulted  in  a  substantial  increase  in  the 
change  in  over  most  recent  record-fire  score  alone.  The  additional  variance 
accounted  for  by  basic  marksmanship  knowledge  and  trait  worry  was  negligible. 
Overall,  the  largest  contributors  to  the  prediction  of  record-fire  score  were 
perceptual-motor  variables.  Of  these  variables,  the  use  of  firing  line  experience 
contributed  more  to  the  prediction  overall  when  used  in  conjunction  with  basic 
marksmanship  knowledge  and  trait  worry.  While  the  cognitive  and  affective 
variables  accounted  for  very  little  of  the  variance,  their  combination  increased  the 
amount  of  variance  explained  by  the  perceptual-motor  variables.  That  is,  was  .15 
{R  =  .39)  when  the  predictors  were  only  most  recent  record-fire  score  and  firing  line 
experience. 

Table  45 


Incremental  Validity  Analyses  (n  =  95) 


Variable 

Regression  statistic 

R 

jR2 

Adj.  R2 

SE 

AR2 

p  value 

Most  recent  record-fire  score 

.38 

.13 

.12 

7.72 

.13 

<.000 

Firing  line  experience  (pre-qual.) 

.51 

.26 

.24 

7.15 

.13 

<.000 

Basic  marksmanship  knowledge 

.51 

.26 

.24 

7.19 

.00 

.805 

Trait  worry 

.52 

.27 

.24 

7.17 

.01 

.241 

Sensitivity  to  group  differences.  To  test  the  sensitivity  of  our  measures,  t  tests 
were  conducted  between  low  performers  (defined  using  the  same  criteria  as  in 
Study  1)  and  others  (i.e.,  non-low  performers).  As  shown  in  Table  46,  significant 
differences  were  found  between  low  performers  and  others  on  nearly  all  perceptual- 
motor  variables,  with  low  performers  showing  lower  performance  on  the  various 
measures  than  others.  Similarly,  low  performers  were  significantly  lower  than  others 
on  measures  of  basic  marksmanship  knowledge,  knowledge  map,  and  self-reported 
level  of  knowledge  of  the  fundamentals  of  marksmanship.  Finally,  low  performers 
had  significantly  higher  scores  on  the  trait  worry  measure  (indicating  more  worry 
about  qualification  in  general),  and  higher  scores  on  the  state  anxiety  measure 
(indicating  higher  anxiety  when  asked  to  predict  how  they  would  feel  during 
qualification  [1  to  3  days  into  the  future]). 
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These  results  support  the  classification  scheme,  and  provide  evidence  of  the 
sensitivity  of  the  basic  knowledge  measure  and  knowledge  map  measure  to 
differences  in  experience. 

A  second  set  of  comparisons  on  our  measures  were  conducted  between 
participants  who  had  just  completed  the  coaches  course  and  participants  in  the  SLR 
training.  Of  interest  are  comparisons  on  the  knowledge  measures  —  presumably, 
participants  who  had  just  completed  the  coaches  course  should  perform  higher  on 
our  knowledge  measures  than  SLR  participants.  The  coaches  course  trains  Marines 
to  be  rifle  marksmanship  coaches  and  covers  the  fundamentals  in  much  greater 
depth  than  the  classroom  training  SLR  Marines  receive. 
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Table  46 


t  tests  of  Cognitive  Measures  Between  Low  Performers  and  Others 


Low 

Others 

Measure 

n 

M 

SD 

n 

M 

SD 

p  value 

Perceptual-motor 

Record-fire  score 

13 

26.92 

8.63 

112 

40.41 

0.66 

<.001 

Most  recent  record-fire  score 

14 

25.57 

7.25 

121 

39.01 

0.75 

<.001 

Second  most  recent  record-fire 

5 

25.6 

0.89 

84 

39.68 

0.91 

.001 

score® 

Third  most  recent  record-fire 

4 

25.25 

2.63 

54 

40.28 

1.24 

.002 

score® 

Frequency  of  shooting  outside  job 

15 

1.8 

1.32 

121 

2.55 

0.13 

.052 

Years  of  shooting  experience 
before  joining  Marines 

13 

1.38 

4.17 

120 

4.28 

0.48 

.021 

Firing  line  experience  (pre-qual.) 

18 

2.38 

.64 

122 

2.99 

.55 

<.001 

Firing  line  experience  (post-qual.)® 

6 

2.86 

.25 

27 

3.43 

.33 

<.001 

Cognitive 

Basic  marksmanship  knowledge 

12 

24.33 

3.85 

96 

27.67 

0.43 

.011 

Shot  group  depiction 

15 

2 

1.25 

120 

1.99 

0.1 

.942 

Evaluation  of  shooter  positions 

14 

15.64 

5.18 

112 

16.72 

0.61 

.419 

Knowledge  map 

15 

1.27 

2.28 

114 

3.74 

0.5 

.038 

Perceived  level  of  marksmanship 
knowledge 

15 

2.8 

0.75 

118 

3.31 

0.06 

.008 

Perceived  utility  of  marksmanship 
knowledge 

15 

1.62 

0.49 

119 

1.65 

0.04 

.798 

Affective 

Trait  worry 

13 

2.27 

0.94 

117 

1.68 

0.06 

.018 

State  worry  (pre-qual.) 

15 

1.78 

0.74 

95 

1.56 

0.06 

.212 

State  worry  (post-qual.)® 

7 

2.08 

0.44 

44 

1.7 

0.1 

.059 

State  anxiety  (pre-qual.) 

15 

2.39 

0.79 

96 

1.86 

0.07 

.010 

State  anxiety  (post-qual.)® 

7 

2 

0.83 

43 

1.69 

0.12 

.224 

^Non-parametric  test  (Kruskal-Wallis). 

As  shown  in  Table  47,  significant  differences  were  observed  in  the  shot  group 
depiction  and  the  knowledge  map.  Unfortunately,  the  participants  in  the  coaches 
course  were  unavailable  to  take  the  basic  marksmanship  knowledge  assessment.  The 
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most  remarkable  difference  is  seen  in  the  knowledge  map  scores.  Participants  in  the 
coaches  course  on  average  scored  almost  3.5  propositions  higher  than  SLR 
participants.  This  difference  is  interesting  because  it  is  consistent  with  the  idea  that 
the  knowledge  map  measures  conceptual  knowledge  —  the  coaches  course 
curriculum  emphasizes  cause-effect  relations  among  position,  aiming,  trigger,  and 
breathing  topics,  presumably  what  knowledge  mapping  is  suited  to  capture. 


Table  47 

t  tests  of  Cognitive  Measures  Between  Marines  in  the  Coaches  Course  and  Participants 


SLR  Coaches  course 


« 

M 

SD 

n 

M 

SD 

p  value 

Perceptual-motor 

Most  recent  record-fire  score 

139 

37.19 

9.6 

15 

44.87 

2.39 

.004 

Second  most  recent  record-fire  score 

89 

38.89 

8.71 

13 

46.23 

1.88 

.004 

Third  most  recent  record-fire  score 

58 

39.24 

9.64 

11 

48.64 

1.46 

.002 

Frequency  of  shooting  outside  job 

145 

2.47 

1.43 

15 

2.6 

0.34 

.734 

Years  of  shooting  experience  before 
joining  Marines 

142 

4.05 

5.24 

15 

2.73 

1.48 

.361 

Firing  line  experience  (pre-qual.) 

143 

2.92 

.60 

15 

3.23 

.47 

.031 

Cognitive 

Shot  group  depiction^* 

143 

1.98 

1.15 

15 

3.07 

0.25 

.001 

Evaluation  of  shooter  positions 

141 

16.72 

6.18 

14 

17.21 

1.64 

.774 

Knowledge  map^ 

143 

3.45 

5.12 

15 

6.87 

1.82 

.004 

Perceived  level  of  marksmanship 
knowledge 

143 

3.26 

0.65 

15 

3.52 

0.12 

.127 

Perceived  utility  of  marksmanship 
knowledge 

144 

1.65 

0.44 

15 

1.5 

0.09 

.200 

Affective 

Trait  worry 

139 

1.77 

0.75 

15 

1.45 

0.60 

.109 

^Non-parametric  test  (Kruskal-Wallis). 

Sensitivity  to  instruction.  The  final  set  of  analyses  examined  differences 
between  a  posttest  and  pretest  of  the  measures  administered  to  the  same  sample  of 
participants.  As  shown  in  Table  48,  there  were  significant  gains  over  the  period 
spanning  classroom  training  (Phase  I),  live-fire  practice,  and  qualification  (Phase  II) 
on  basic  marksmanship  knowledge  and  knowledge  mapping.  Interestingly,  the 
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differences  in  knowledge  map  scores  increased  over  occasion.  There  was  an  average 
of  6.25  more  correct  propositions  in  the  posttest  map  on  the  fifth  occasion  over  the 
initial  map.  Similarly,  there  was  an  average  of  1.8  propositions  gained  over  the 
classroom  training  period.  Note  that  the  knowledge  mapping  tasks  were  slightly 
different.  On  the  fifth  occasion,  participants  recreated  their  maps  from  scratch 
whereas  on  the  third  occasion,  participants  made  changes  to  an  existing  map  (i.e., 
their  knowledge  map  from  the  previous  mapping  occasion). 

Table  48 


Paired-Sample  Tests  of  Mean  Differences  Between  Measures  Administered  Before  Classroom 
Training  and  After  Qualification  Attempt 


n 

M 

Pretest 

SD 

SE 

M 

Posttest 

SD 

SE 

Mean 

differ¬ 

ence 

p  value 

Basic  marksmanship 
knowledge 

22 

26.91 

4.25 

0.91 

28.77 

3.94 

0.84 

1.86 

.002« 

Shot  group  depiction 

33 

2.06 

1.14 

0.20 

2.12 

1.02 

0.18 

0.06 

.853'’ 

Evaluation  of  shooter 
positions 

29 

15.41 

5.98 

1.11 

15.62 

6.31 

6.31 

-.21 

.630 

Knowledge  map 
(occasion  5  vs.  1) 

16 

4.88 

8.09 

2.02 

11.10 

12.97 

3.24 

6.25 

.004^ 

Knowledge  map 
(occasion  3  vs.  1) 

31 

6.48 

8.25 

1.48 

4.65 

6.98 

1.25 

1.84 

<.001« 

^Paired  t  test.  '’Non-parametric  test  (paired  Wilcoxon). 


Summary  and  Discussion  of  Study  2 

Reliability  of  measures.  Overall,  our  measures  demonstrated  moderate  to  high 
reliabilities.  The  lowest  reliability  was  with  the  shot  group  depiction  task  (a  =  .31), 
similar  to  Study  1.  The  basic  marksmanship  knowledge  measure  had  a  moderate  a 
(.73).  The  survey  measures  (i.e.,  affective,  firing  line  experience,  perceived  level  of 
marksmanship  knowledge,  perceived  utility  of  marksmanship  knowledge) 
demonstrated  high  reliabilities,  with  a  in  the  .8  to  .9  range.  The  exception  to  this  was 
firing  line  experience  on  qualification  day  (a  =  .58).  The  knowledge  map  measures 
also  demonstrated  high  reliabilities  when  scores  from  three  expert  criterion  maps 
were  used  as  items  (a  =  .92). 
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Predictive  validity.  The  best  predictor  of  record-fire  score  was  firing  line 
experience  administered  after  the  participants  qualified  (r  =  .57,  p  <  .01).  When  this 
measure  was  administered  prior  to  qualification,  the  relationship  was  lower  (r  =  .33, 
p  <  .01),  comparable  to  participants'  self-reported  most  recent  record-fire  score  (r  = 
.34,  p  <  .01).  Affective  variables  correlated  negatively  with  record-fire  score,  ranging 
from  -.1  to  -.4.  In  general,  the  magnitude  of  the  correlations  was  higher  when  the 
measure  was  administered  after  participants  attempted  to  qualify.  Interestingly,  trait 
worry  was  a  moderately  good  negative  predictor  of  performance  (r  =  -.29,  p  <  .01). 
With  respect  to  cognitive  predictors  of  record-fire  score,  only  the  perceived  level  of 
marksmanship  knowledge  predicted  record-fire  score  (r  =  .26,  p  <  .05). 

Construct  validity.  Overall,  the  sample  in  Study  2,  compared  to  Study  1,  was 
older  and  more  experienced  in  shooting.  For  this  reason,  we  expected  this  sample  to 
be  slighter  further  along  in  skill  development.  Compared  to  a  novice  sample,  in  a 
more  experienced  sample  the  skill-development  model  (Ackerman,  1987, 1992;  Fitts 
&  Posner,  1967)  suggests  that  cognitive  variables  (e.g.,  aptitude  and  content-related 
abilities  and  presumably  knowledge)  have  less  influence  on  performance,  and 
perceptual  and  motor  variables  would  have  more  impact  on  performance.  We 
considered  prior  shooting  scores  and  other  shooting-related  variables  as  proxies  for 
the  perceptual-motor  construct. 

Overall,  there  were  no  significant  relationships  between  record-fire  scores  and 
measures  of  knowledge.  Only  perceived  level  of  marksmanship  knowledge  was 
related  to  record-fire  score  (r  =  .26,  p  <  .05).  With  respect  to  perceptual-motor 
variables,  record-fire  scores  were  related  with  most  recent  record-fire  score  (r  =  .34,  p 
<  .01)  and  prior  firing  line  experience  (r  =  .33,  p  <  .01).  Interestingly,  there  was  a  low 
but  significant  relationship  between  basic  marksmanship  and  prior  firing  line 
experience  (r  =  .19,  p  <  .05).  This  set  of  relationships  are  consistent  with  the  skill- 
development  model  that  suggests  that  content-related  abilities  are  less  influential  on 
performance  compared  to  perceptual-motor  variables  for  more  experienced 
shooters. 

The  correlations  among  record-fire  score,  firing  line  experience,  state  anxiety, 
and  state  worry  were  consistent  with  expectations.  The  direction  of  the  relationship 
and  the  large  magnitudes  suggest  that  the  measures  were  working  as  intended. 
Firing  line  experience  was  negatively  associated  with  state  anxiety  and  state  worry. 
Participants  who  reported  more  positive  shooting  experience  also  reported  lower 
anxiety  and  worry.  Similarly,  participants  with  more  anxiety  also  reported  more 
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worry.  Trait  worry  was  correlated  sigyiificantly  and  moderately  with  firing  line 
experience  {negatively),  state  worry  (positively),  and  state  anxiety  (positively),  a 
result  that  also  is  consistent  with  expectations. 

Sensitivity  of  measures.  Low  performers,  compared  to  others,  scored  lower  on 
current  and  prior  record-fire  scores,  reported  less  shooting  experience  prior  to 
joining  the  Marines,  and  reported  poorer  firing  line  experiences.  In  addition,  they 
scored  lower  on  the  basic  marksmanship  knowledge  and  knowledge  map  measures, 
consistent  with  expectations.  There  were  no  differences  on  the  shot  group  depiction 
and  evaluation  of  shooter  positions  performance  measures.  Low  performers  also 
reported  more  trait  worry  and  pre-qualification  anxiety  than  others. 

When  the  SLR  sample  was  compared  to  participants  who  just  completed  the 
coaches  course,  coaches  scored  higher  on  all  previous  record-fire  scores  and 
reported  more  positive  firing  line  experiences.  Coaches  also  scored  higher  on  the 
knowledge  map  and  shot  group  depiction  performance  measures.  There  was  no 
difference  between  the  samples  on  the  evaluation  of  shooter  positions  performance 
measure,  and  due  to  time  restrictions,  the  coaches  did  not  take  the  basic 
marksmanship  knowledge  measure. 

Study  3— Method 

The  purpose  of  Study  3  was  to  test  our  measures  on  entry-level  officers  (i.e., 
2nd  Lieutenants  [LT]).  Also,  we  wanted  to  replicate  the  findings  from  Study  2 
regarding  firing  line  experience,  knowledge  map  sensitivity,  and  anxiety  measures. 
We  also  pilot  tested  new  measures. 

The  second  aspect  of  this  sample  was  that  the  participants  were  entry-level  and 
thus  had  little  or  no  prior  shooting  or  marksmanship  training.  We  could  thus  focus 
on  the  prediction  of  shooting  performance  based  solely  on  our  measures  without  the 
confounding  factor  of  prior  shooting  experience. 

Participants 

Fifty-three  entry-level  2nd  LT  Marines  participated  in  the  study  at  WTBN 
Quantico,  VA.  A  description  of  the  sample  is  shown  in  Table  49.  In  general,  the 
Marines  were  male  (88%)  with  little  prior  shooting  experience. 
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Table  49 

Sample  Descriptive  Statistics 

Variable 

n 

M 

SD 

Min. 

Max. 

Age 

53 

23.70 

1.78 

21 

28 

Years  in  service 

48 

0.63 

1.00 

0 

6 

Frequency  of  shooting  as  part  of  duties^ 

53 

1.32 

0.78 

1 

4 

Frequency  of  shooting  outside  of  duties® 

53 

1.98 

1.03 

1 

5 

Years  of  shooting  experience  before  Marines 

52 

2.25 

3.95 

0 

15 

^1  =  Never,  2  =  Once  or  twice,  3  =  A  few  times,  4  =  Often,  5  =  Very  often. 


As  part  of  the  normal  training  process,  participants  receive  about  two  days  of 
classroom  instruction  on  the  fundamentals  of  rifle  marksmanship.  The  next  day  is 
spent  "zeroing"  the  rifle  (calibrating  the  rifle  sights  for  a  particular  distance).  The 
following  five  days  are  spent  on  the  firing  range.  The  first  four  of  these  days  are  for 
live-fire  practice.  The  last  day  is  reserved  for  qualification  trials.  During  practice, 
participants  could  get  help  from  range  coaches  (the  ratio  of  coaches  to  students  was 
1  to  3  or  4  students).  During  qualification,  participants  received  no  help.  If 
participants  failed  their  first  qualification  attempt,  they  were  provided  with 
additional  coaching  and  given  the  opportunity  to  attempt  to  qualify  again  in 
subsequent  days.  Marines  who  failed  the  second  qualification  attempt  were  at  risk  of 
being  dropped  from  the  Marine  Corps.  All  Marines  in  the  current  study  qualified. 

Design 

Three  groups  of  participants  were  adnrinistered  tasks  over  a  two-week  period. 
Groups  2  and  3  differed  by  type  of  knowledge  mapping  task.  Group  3  was  provided 
with  the  same  set  of  links  as  Group  2,  but  in  addition,  they  had  the  option  to  type  in 
their  own  links.  There  was  only  one  instance  of  this  occurring  across  all 
administrations  of  the  tasks;  thus,  we  collapsed  the  sample  into  a  single  group.  We 
tested  an  instructional  intervention  in  Group  1  on  one  day  but  otherwise.  Group  1 
was  identical  to  Group  2. 

Measures 

The  following  measures  were  provided  to  us  by  the  USMC  or  administered  to 
Marines: 

Record-fire  scores.  Official  qualification  scores  were  provided  to  us  by  WTBN. 
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Knowledge  mapping.  The  set  of  terms  and  links  and  scoring  method  remained 
the  same  as  Study  2. 

Knowledge  mapping  of  problem  solving  (shot-to-shot).  The  task  is  given  in 
Appendix  T.  This  measure  was  included  for  pilot  testing  purposes  (usability)  and 
will  not  be  discussed  further. 

Knowledge  mapping  of  procedural  knowledge  (data  book  procedure).  The 
task  is  given  in  Appendix  U.  This  measure  was  included  for  pilot  testing  purposes 
(usability)  and  will  not  be  discussed  further. 

Basic  marksmanship  knowledge.  We  used  a  revised  measure  based  on  Study 
2.  The  format  of  this  measure  was  paper.  The  revised  measure  is  given  in  Appendix 

V. 

General  Classification  Test  (GCT)  scores.  Official  GCT  scores  were  provided 
to  us  by  the  USMC.  This  test  is  administered  only  to  commissioned  and  warrant 
officers.  GCT  is  used  as  a  measure  for  aptitude. 

Classroom  Test  of  Scientific  Reasoning.  Lawson's  Classroom  Test  of  Scientific 
Reasoning  (revised  24-item  multiple  choice  edition)  was  used  to  measure  scientific 
reasoning  (Lawson,  1987,  2000).  This  measure  was  included  for  pilot  testing 
purposes  (usability)  and  will  not  be  discussed  further.  The  task  is  given  in  Appendix 

W. 

Shot  group  depiction  task.  We  used  the  same  measure  from  Study  2.  The 
format  of  this  measure  was  paper. 

State  anxiety  survey.  We  used  the  same  set  of  measures  as  Study  2  to  measure 
participants'  state  anxiety  about  their  shooting  during  practice  days  and  during 
qualification.  The  format  of  this  measure  was  paper. 

State  worry.  We  used  the  same  set  of  measures  as  Study  2  to  measure 
participants'  worry  about  their  shooting  during  practice  days  and  during 
qualification.  The  format  of  this  measure  was  paper. 

Firing  line  survey.  This  was  a  revised  measure  that  we  used  to  gather 
information  on  participants'  overall  experience  on  the  firing  line.  The  format  of  this 
measure  was  paper. 

Perceived  level  of  marksmanship  knowledge.  We  used  the  same  measure  as 
Study  2. 
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Perceived  utility  of  marksmanship  knowledge.  We  used  the  same  measure  as 
Study  2. 

Background  survey.  We  used  the  same  measure  from  Study  2  with  slight 
modifications  to  adjust  for  the  new  (i.e.,  entry-level)  sample.  The  format  of  this 
measure  was  paper. 

Evaluation  of  shooter  positions  (ESP).  We  revised  the  software  from  Study  2 
given  comments  from  WTBN.  WTBN  supervised  the  positioning  of  the  Marine 
model.  In  addition,  new  doctrine  specified  the  use  of  the  loop  sling,  which  was  not 
reflected  in  the  Study  2  ESP  task.  The  scoring  rubric  is  given  in  Appendix  X. 

As  with  the  previous  version  of  ESP,  the  task  for  the  Marine  was  to  diagnose 
problems  with  the  shooter.  The  participant  was  presented  with  a  shooter  (as  shown 
in  Figure  15).  The  figure  was  a  QuickTime  VR  image,  so  the  participant  could  rotate 
the  image  and  have  a  360-degree  view.  The  participant  was  required  to  judge  the 
extent  to  which  each  body  element  was  in  its  proper  position,  and  indicate  this 
judgment  by  selecting  the  radio  button  for  each  position  element  (lower  left  corner 
of  Figure  15). 

If  the  participant  selected  any  option  other  than  "proper,"  the  participant  was 
required  to  fix  the  position  element.  Clicking  on  the  "Correct  It"  button  opened  a 
video  window.  The  video  was  of  a  Marine  exercising  the  full  range  of  motion  that 
spanned  correct  and  incorrect  positions.  The  participant  was  required  to  use  the 
slider  bar  to  indicate  the  correct  position. 
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Figure  15.  Revised  ESP  task  user  interface. 

Reliability  of  measures.  Table  50  shows  the  reliabilities  of  the  measures.  In 
general,  the  measures  showed  high  reliabilities.  As  in  Study  1  and  Study  2,  the  shot 
group  measure  showed  very  low  reliabilities.  The  basic  knowledge  measure  showed 
a  decrease  in  reliability  from  Study  2. 


tl'g|'  pei<t| 

^ ewe  .  . . . . - . .  . . 
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Table  50 


Reliability  of  Measures 


Measure 

n 

Number 
of  items 

Cronbach's 

alpha 

Basic  marksmanship  knowledge 

48 

35 

.64 

Perceived  level  of  marksmanship  knowledge 

53 

5 

.93 

Perceived  utility  of  marksmanship  knowledge 

53 

6 

.76 

Knowledge  map 

Day  1 

50 

3 

.91 

Day  2 

45 

3 

.92 

Day  3 

46 

3 

.92 

Day  4 

44 

3 

.91 

Day  5  (qualification  day) 

45 

3 

.92 

Day  6 

50 

3 

.93 

Knowledge  map  (shot  to  shot) 

52 

2 

.98 

Classroom  test  of  scientific  reasoning 

44 

22 

.84 

Shot  group  depiction 

49 

5 

.23 

State  worry 

Day  1 

53 

6 

.67 

Day  3 

51 

6 

.82 

Day  5  (qualification  day) 

49 

6 

.82 

State  anxiety 

Day  1 

52 

5 

.87 

Day  3 

51 

5 

.88 

Day  5  (qualification  day) 

50 

5 

.93 

Firing  line  experience 

Day  1 

53 

6 

.78 

Day  2 

52 

6 

.81 

Day  3 

53 

6 

.88 

Day  4 

53 

6 

.83 

Day  5  (qualification  day) 

50 

6 

.84 
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Procedure 

Measures  were  administered  to  participants  in  the  order  and  allotted  times 
shown  in  Table  51.  In  general,  participants  completed  the  tasks  well  within  the  times 
listed  and  were  given  more  time  if  needed.  Because  of  a  technical  problem,  the 
evaluation  of  shooter  positions  measure  was  rescheduled  from  the  first  occasion  to 
the  second  occasion. 
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KM  =  knowledge  mapping  task  (revise  previous  knowledge  map);  CTSR  =  classroom  test  of  scientific  reasoning;  ESP  =  evaluation  of  shooter 
positions.  ATT  =  attitudes  toward  marksmanship  knowledge;  KMSS  =  knowledge  mapping  (shot-to-shot  explanation);  KMDB  =  knowledge 
mapping  (data  book  procedure);  PF  =  post-fire  surveys  (firing  line  experience,  state  anxiety,  state  worry);  KM-P  =  post-training  knowledge 
mapping  (create  a  new  knowledge  map);  BMK-P  =  post-training  basic  marksmanship  knowledge  test;  ESP-P  =  post-training  ESP;  ONT  =  ontology 
pilot  test  (reported  in  Chung,  Delacruz,  Dionne,  <&  Bewley,  2003). 
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Study  3— Results 


Preliminary  Analyses 

Prior  to  conducting  the  main  analyses,  we  checked  for  group  differences  on  the 
record-fire  scores  and  on  the  knowledge  measures.  Separate  one-way  ANOVAs 
were  conducted  to  check  for  differences  across  groups  on  (a)  record-fire  scores,  (b) 
knowledge  mapping  scores,  (c)  basic  marksmanship  knowledge  scores,  (d)  shot 
group  depiction  scores,  and  (e)  ESP  scores. 

There  were  no  significant  differences  on  GCT,  ESP,  basic  marksmanship 
knowledge,  knowledge  mapping  pretest,  frequency  of  shooting  outside  of  duties, 
and  years  of  shooting  experience  prior  to  joining  the  Marines.  There  was  a 
significant  difference  on  the  record-fire  scores  (f  (2,50)  =  5.96,  MSE  =  63.8,  p  =  .005). 
Pair-wise  comparisons  indicated  Group  2  performed  significantly  higher  (M  =  43.7, 
SD  =  8.5,  n  =  18)  than  Group  1  (M  =  34.5,  SD  =  8.02,  n  =  17)  and  Group  3  (M  =  37.6, 
SD  =  7.5,  n  =  18).  A  review  of  the  range  conditions  during  qualification  indicated 
that  Group  1  participants  qualified  in  a  severe  thunderstorm.  In  addition.  Group  1 
received  an  instructional  intervention.  Thus,  Group  1  was  dropped  from  the 
analyses  reported  in  this  section.  Table  52  shows  descriptive  statistics  for  the  revised 
sample. 


Table  52 

Revised  Sample  Descriptive  Statistics 


Variable 

n 

M 

SD 

Min. 

Max. 

Age 

36 

23.5 

1.8 

21 

28 

Years  in  service 

34 

0.70 

1.14 

0 

6 

Frequency  of  shooting  as  part  of  duties" 

36 

1.11 

.52 

1 

4 

Frequency  of  shooting  outside  of  duties" 

36 

1.92 

.94 

1 

4 

Years  of  shooting  experience  before  Marines 

36 

2.17 

3.92 

0 

15 

"1  =  Never,  2  =  Once  or  twice,  3  =  A  few  times,  4 

=  Often,  5  = 

Very  often. 

Main  Analyses 

Table  53,  Table  55,  and  Table  58  show  descriptive  statistics  of  the  perceptual- 
motor,  cognitive,  and  affective  variables.  Table  54,  Table  56,  Table  57,  Table  59,  and 
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Table  60  present  intercorrelations  and  correlations  among  the  three  groups  of 
variables.  Interestingly,  the  mean  record-fire  score  for  the  entry-level  participants  is 
similar  in  magnitude  with  our  prior  sustainment-level  samples.  In  general,  as 
indicated  by  the  firing  line  experience  surveys,  participants  reported  increasingly 
positive  firing  line  experience  as  the  week  went  on,  with  the  most  positive  firing  line 
experience  reported  on  qualification  day.  Additional  analyses  conducted  on  the 
shooting  scores  over  the  live-fire  period  in  general  show  a  positive  fast  and  rapid 
improvement  in  shooting  scores  at  the  beginning  of  the  week,  with  shooting  scores 
improving  at  an  increasingly  slower  rate.  A  detailed  analysis  is  reported  in 
Appendix  Y. 

Table  53 


Descriptive  Statistics  for  Perceptual-Motor  Variables 


Variable 

n 

M 

SD 

Min. 

Max. 

Record-fire  scored" 

36 

40.64 

8.44 

25 

57 

Frequency  of  shooting  outside  job^’ 

36 

1.92 

0.94 

1 

4 

Years  of  shooting  experience  before  joining  Marines 

36 

2.17 

3.92 

0 

15 

Firing  line  experience'^ 

Live-fire  practice  Day  1 

36 

2.48 

.42 

1.67 

3.67 

Live-fire  practice  Day  2 

35 

2.62 

.54 

1.67 

3.83 

Live-fire  practice  Day  3 

36 

2.88 

.65 

1.33 

4 

Live-fire  practice  Day  4 

36 

3.07 

.59 

1.67 

4 

Qualification  day 

34 

3.18 

.61 

2 

4 

Note.  ELR  record-fire  scores  were  converted  to  SLR  equivalent  scores.  See  Appendix  G  for 
the  conversion  table. 

afl  -  24  =  unqualified,  25  -  34  =  marksman,  35  -  39  =  sharpshooter,  40  -  65  =  expert.  ’^1  = 

Never,  2  =  Once  or  twice,  3  =  A  few  times,  4  =  Often,  5  =  Very  often.  <^1  =  Almost  never,  2  = 

Sometimes,  3  =  Often,  4  =  Almost  always  (higher  values  indicate  higher  positive  experience). 

Intercorrelations  among  the  perceptual-motor  variables  (see  Table  54)  show 
significant  relationships  among  record-fire  score  and  firing  line  experience.  The  best 
predictor  of  record-fire  score  is  the  firing  line  measure  administered  after  the 
participants  qualified,  as  in  Study  2.  Interestingly,  in  general,  the  previous  day's 
firing  line  experience  was  not  always  a  predictor  of  the  next  day's  experience, 
although  each  day's  firing  line  experience  was  predictive  of  record-fire  scores. 
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Table  54 

Intercorrelations  (Pearson)  Among  Perceptual-Motor  Variables 

Variable 

1 

2 

3 

4 

5 

6  7 

1.  Record-fire  score 

- 

2.  Frequency  of  shooting  outside  job 

.30 

- 

3.  Years  of  shooting  experience  before  joining 

.26 

.71** 

- 

Marines 

4.  Firing  line  experience,  live-fire  practice  Day  1 

.35* 

.16 

.25 

- 

5.  Firing  line  experience,  live-fire  practice  Day  2 

.51* 

.23 

-.01 

.30 

- 

6.  Firing  line  experience,  live-fire  practice  Day  3 

.34* 

.16 

.11 

.10 

.62** 

- 

7.  Firing  line  experience,  live-fire  practice  Day  4 

.49* 

.08 

.03 

.42* 

.55** 

.38* 

8.  Firing  line  experience,  qualification  day 

.20 

.05 

.38* 

.46* 

.33  .29 

With  respect  to  cognitive  variables  (Table  55),  participants  in  general  reported 
they  had  a  moderate  amount  of  knowledge  of  the  fundamentals  of  marksmanship, 
and  they  perceived  the  value  of  knowledge  of  the  fundamentals  to  shooting  well. 
Further,  when  tested  with  our  knowledge  measures,  performance  on  the  basic 
knowledge  measure  was  moderately  high  (mean  score  was  80%  correct). 
Interestingly,  performance  on  the  shot  group  depiction  task  was  similar  to  the  Study 
2  sample.  When  the  participants  in  Study  3  were  compared  to  the  SLR  and  coaches 
course  participants  in  Study  2,  the  ELR  officers  performed  significantly  higher  than 
both  groups.  The  mean  difference  between  the  ELR  and  SLR  samples  was  on 
average  9.9  propositions,  f(58.12)  =  6.69,  p  <  .001,  and  the  mean  difference  with  the 
participants  in  the  coaches  course  was  6.5  propositions,  f(63)  =  2.33,  p  =  .02. 
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Table  55 

Descriptive  Statistics  for  Cognitive  Variables 

Variable 

n 

M 

SD 

Min. 

Max. 

General  Classification  Test 

36 

124.14 

10.78 

100 

150 

Basic  marksmanship  knowledge^ 

34 

28.32 

3.00 

22 

33 

Shot  group  depiction’’ 

33 

2.06 

0.93 

0 

4 

Evaluation  of  shooter  positions  (pre-live  fire)'’ 

33 

42.33 

3.48 

37 

50 

Evaluation  of  shooter  positions  (post-qualification)” 

36 

43.69 

3.55 

31 

49 

Knowledge  map 

Pre-classroom  training 

34 

7.76 

7.19 

0 

24 

Post-classroom  training 

28 

8.93 

7.17 

0 

27 

Live-fire  practice  Day  1 

29 

9.34 

7.47 

0 

27 

Live-fire  practice  Day  3 

29 

10.24 

8.30 

0 

28 

Qualification  day 

31 

9.65 

7.85 

0 

26 

Post-qualification 

33 

12.76 

8.39 

0 

27 

Perceived  level  of  marksmanship  knowledge,  post-classroom 

training”’ 

36 

3.00 

0.42 

2.2 

3.8 

Perceived  utility  of  marksmanship  knowledge,  post-classroom 

training” 

36 

1.39 

0.36 

1 

2.17 

^Maximum  possible  is  35.  ’’Maximum  possible  is  5.  ‘’Administered  at  the  beginning  of  live-fire 
practice  and  after  live-fire  practice.  =  Not  at  all,  2  =Some,  3  =  Moderate  amount,  4  =  Very  much  (higher 
values  indicate  more  self-reported  knowledge).  <’1  =  Strongly  agree,  2  =  Agree,  3  ^Disagree,  4  =  Strongly 
disagree  (higher  values  indicate  lower  perceived  utility  of  marksmanship  knowledge). 

Intercorrelations  among  the  cognitive  variables  shown  in  Table  56  suggest  that 
the  knowledge  measures  were  in  part  operating  as  intended.  Basic  marksmanship 
knowledge  was  significantly  related  to  aptitude  and  self-reported  level  of 
knowledge.  These  results  are  consistent  with  the  idea  that  the  capacity  to  learn 
mediated  how  much  was  learned. 
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The  most  interesting  results  were  with  the  knowledge  maps.  The  mean  scores 
shown  in  Table  55  show  slow,  positive  changes  over  occasions.  The  pre-classroom¬ 
training  knowledge  map  correlated  significantly  with  all  subsequent  knowledge 
maps.  The  occasion-to-occasion  correlation  between  map  scores  (i.e.,  where 
participants  were  revising  their  maps  from  the  previous  occasion)  was  consistently 
high.  The  magnitude  dropped  considerably,  however,  when  participants  were 
retested  after  qualification  (i.e.,  participants  were  given  a  blank  map).  This  drop  in 
magnitude  suggests  that  learning  occurred  differentially  over  the  course  of 
classroom  training  and  live-fire  practice.  The  map  scores  increased  on  average  by  5 
propositions,  and  the  correlation  between  the  pretest  and  posttest  suggests  that 
changes  were  less  than  uniform  across  participants. 

Occasion-to-occasion  map  scores  were  very  consistent,  with  correlations  in  the 
mid  .90s.  This  result  suggests  that  there  were  either  few  changes  or  the  changes  that 
were  made  were  inconsequential  (i.e.,  resulted  in  little  changes  in  the  scores).  The 
most  interesting  occasion-to-occasion  change  was  between  the  knowledge  map 
scores  on  qualification  day  and  the  map  scores  on  the  posttest.  The  correlation  in 
this  case  dropped  to  .50. 

Bivariate  correlations  between  the  perceptual-motor  variables  and  cognitive 
variables  (see  Table  57)  show  significant  and  moderate  relationships  between 
record-fire  score  and  a  variety  of  aptitude  and  knowledge  measures.  The  highest 
correlation  with  record-fire  score  was  GCT,  followed  by  participants'  self-ratings  of 
how  much  they  know  about  the  fundamentals,  and  the  basic  knowledge  of 
marksmanship  measure.  The  self-rating  measure  also  correlated  with  firing  line 
experience  during  qualification  day  and  with  two  other  live-fire  practice  days. 
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Table  57 

Correlations  Between  Perceptual-Motor  Variables  and  Cognitive  Variables 


Perceptual-motor  variable 


Variable 

RF 

FSOJ 

YEXP 

FLEl 

FLE2 

FLE3 

FLE4 

FLEQ 

1.  General  classification  test 

.45* 

.25 

.17 

-.18 

.26 

.17 

.28 

.16 

2.  Basic  marksmanship 
knowledge 

.38* 

.13 

.21 

.05 

.28 

.36* 

.31 

.20 

3.  Shot  group  depiction 

.11 

.28 

.12 

.14 

.11 

.04 

.37* 

-.22 

4.  Evaluation  of  shooter 
positions  (pre-live  fire) 

-.38* 

-.20 

.00 

-.24 

-.13 

-.03 

-.01 

-.42* 

5.  Evaluation  of  shooter 
positions  (post-qual.) 

-.13 

-.18 

-.09 

.12 

-.23 

-.12 

.07 

-.13 

6.  Knowledge  map,  pre¬ 
classroom  training 

.29 

.31 

.06 

-.19 

.29 

.22 

.21 

.13 

7.  Knowledge  map,  post- 
classroom  training 

.25 

.39* 

.04 

-.02 

.44* 

.13 

.39* 

.13 

8.  Knowledge  map,  live-fire 
practice  Day  1 

.19 

.32 

.05 

-.06 

.29 

.10 

.31 

-.01 

9.  Knowledge  map,  live-fire 
practice  Day  3 

.17 

.22 

.09 

-.09 

.31 

.11 

.28 

.07 

10.  Knowledge  map, 
qualification  day 

.10 

.24 

.01 

-.16 

.20 

-.08 

.10 

-.10 

11.  Knowledge  map,  post¬ 
qualification 

.03 

-.02 

-.03 

-.06 

.11 

.19 

.02 

-.05 

12.  Perceived  level  of  know, 
of  the  fundamentals 

.41* 

-.22 

.04 

.22 

.28 

.27 

.43* 

.34* 

13.  Perceived  utility  of 

-.18 

-.03 

.10 

.08 

.04 

-.03 

-.10 

-.02 

marksmanship 

knowledge 

Note.  RF  =  Record-fire  score.  FSOJ  =  Frequency  shooting  outside  of  job.  YEXP  =  Years  of  shooting 
experience  before  joining  Marines.  FLEl-FLEQ  =  Firing  line  experience,  live-fire  practice  Day  1  to 
Day  4,  and  qualification  day. 

With  respect  to  affective  measures  (Table  58),  participants  in  general  reported 
low  worry  and  low  anxiety  throughout  live-fire  practice  and  on  qualification  day. 
These  results  are  consistent  with  participants'  self-reported  positive  firing  line 
experience  shown  in  Table  53.  Interestingly,  there  are  no  appreciable  increases  in 
worry  and  anxiety  on  qualification  day  compared  to  practice  days. 
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Table  58 


Descriptive  Statistics  for  Affective  Variables 


Variable 

n 

M 

SD 

Min. 

Max. 

Trait  worry,  post-qualification® 

35 

3.61 

0.45 

1.88 

4 

State  worry 

Live-fire  practice  Day 

36 

1.69 

0.40 

1 

2.5 

Live-fire  practice  Day  3*’ 

35 

1.68 

0.55 

1 

2.83 

Qualification  day^ 

33 

1.70 

0.63 

1 

3.67 

State  anxiety 

Live-fire  practice  Day  1*’ 

35 

1.99 

0.51 

1 

2.8 

Live-fire  practice  Day  S*’ 

35 

1.94 

0.64 

1 

3.6 

Qualification  day'’ 

33 

1.83 

0.69 

1 

3.4 

=  Almost  never,  2  =  Sometimes,  3  =  Often,  4  =  Almost  always  (higher 
values  indicate  higher  worry),  =  Not  at  all,  2  -Sometimes,  3  = 

Moderately  so,  4  =  Very  much  so  (higher  values  indicate  higher 
worry/ anxiety). 

Intercorrelations  among  the  affective  measures  are  generally  moderate,  with 
scores  on  the  most  recent  occasion  predicting  scores  on  qualification  day 
significantly  and  moderately.  In  general,  worry  and  anxiety  were  consistent  with 
each  other,  especially  on  qualification  day.  Interestingly,  the  relationship  between 
the  trait  worry  measure  and  the  state  worry  and  state  anxiety  measures  increased  in 
general  the  nearer  qualification  day  approached. 

Table  59 


Intercorrelations  Among  Affective  Variables 


Variable 

1 

2 

3 

4 

5 

6 

7 

1. 

Trait  worry,  post-qualification 

~ 

2. 

State  worry,  live-fire  practice  Day  1 

-.36* 

- 

3. 

State  worry,  live-fire  practice  Day  3 

-.62** 

.48* 

- 

4. 

State  worry,  qualification  day 

-.59** 

.26 

.38* 

- 

5. 

State  anxiety,  live-fire  practice  Day  1 

-.19 

.34* 

,01 

.46* 

- 

6. 

State  anxiety,  live-fire  practice  Day  3 

-.40* 

.44* 

.79** 

.31 

.06 

- 

7. 

State  anxiety,  qualification  day 

-.52* 

.15 

.27 

.83** 

.33 

.37* 

~ 
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Bivariate  correlations  between  the  perceptual-motor  variables  and  affective 
variables  (see  Table  60)  show  significant  and  moderate  to  large  relationships.  The 
best  predictors  of  record-fire  score  were  the  state  measures  of  worry  and  anxiety  at 
qualification;  however,  state  worry  and  anxiety  at  practice  day  3  predicted  record- 
fire  score  as  well,  although  at  lower  magnitudes.  Interestingly,  the  trait  worry 
measure  (administered  four  days  after  qualification)  was  as  good  a  predictor  of 
record-fire  scores  as  practice  day  3  worry  and  anxiety  measures. 

The  large  and  significant  correlations  between  the  state  worry  and  anxiety 
measures  and  the  firing  line  experience  measure  suggest  that  these  measures  were 
working  as  intended.  Participants'  negative  affective  states  (i.e.,  anxiety,  worry) 
were  inversely  related  to  firing  line  experience. 


Table  60 

Correlations  Between  Perceptual-Motor  Variables  and  Affective  Variables 


Perceptual-motor  variable 


Variable 

RF 

FSOJ 

YEXP 

FLEl 

FLE2 

FLE3 

FLE4 

FLEQ 

1. 

Trait  worry,  post¬ 
qualification 

-.45* 

-.05 

.09 

-.28 

-.31 

-.36* 

-.36* 

-.58** 

2. 

State  worry,  live-fire 
practice  Day  1 

-.31 

-.26 

-.08 

-.36* 

-.32 

-.41* 

-.21 

-.39* 

3. 

State  worry,  live-fire 
practice  Day  3 

-.47* 

-.06 

.10 

.03 

-.58** 

-.75** 

-.42* 

-.37* 

4. 

State  worry, 
qualification  day 

-.64** 

.08 

.20 

-.36* 

-.36* 

-.19 

-.40* 

-.78** 

5. 

State  anxiety,  live-fire 
practice  Day  1 

-.22 

-.12 

-.20 

-.56** 

-.10 

.07 

-.10 

-.31 

6. 

State  anxiety,  live-fire 
practice  Day  3 

-.52* 

-.12 

-.05 

-.03 

-.58** 

-.78** 

-.56** 

-.38* 

7. 

State  anxiety, 
qualification  day 

-.61** 

-.06 

.02 

-.31 

-.25 

-.25 

-.39* 

_  79** 

Note.  RF  =  Record-fire  score.  FSOJ  =  Frequency  shooting  outside  of  job.  YEXP  =  Years  of  shooting 
experience  before  joining  Marines.  FLEl-FLEQ  =  Firing  line  experience.  Day  1  to  Day  4,  and 
qualification  day. 

*p  <  .05.  **p  <  .01. 
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Prediction  of  Record-Fire  Performance 

To  predict  record-fire  performance,  we  conducted  a  series  of  multiple 
regression  analyses.  Given  that  small  sample  size,  we  limited  the  number  of 
variables  in  the  regression  model  to  at  most  three.  Five  models  were  tested:  (a) 
perceptual-motor  variables;  (b)  cognitive  variables  only;  (c)  affective  variables;  (d) 
cognitive  and  affective  variables;  and  (e)  perceptual-motor,  cognitive,  and  affective 
variables.  As  shown  in  Table  61,  for  each  regression  model  we  also  conducted  an 
incremental  validity  analysis.  Figure  16  shows  confidence  intervals  for  the  predicted 
scores. 

Firing  line  experience,  GCT,  and  state  worry  were  the  strongest  predictors  in 
their  respective  categories.  For  each  type  of  variable  (i.e.,  perceptual-motor, 
cognitive,  affective),  adding  more  variables  contributed  little  to  the  prediction  of 
record-fire  scores.  When  only  cognitive  and  affective  variables  are  considered  {R  = 
.76),  the  combination  of  aptitude  and  state  worry  is  as  high  as  firing  line  experience. 
The  full  model  combining  all  three  variables  results  in  an  increase  of  R  by  13%  to  .86. 
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Figure  16.  The  95%  confidence  interval  for  the  mean  predicted  record-fire  scores  (n  =  32). 

Sensitivity  to  group  effects.  In  a  series  of  analyses  we  examined  whether 
group  differences  existed  between  participants  who  qualified  expert  and 
participants  who  qualified  as  something  else  (i.e.,  sharpshooter  or  marksman).  Note 
that  most  of  the  participants  reported  having  little  prior  shooting  experience  and 
were  undergoing  basic  training.  Thus,  this  sample  is  relatively  "uncontaminated." 

Table  62  shows  descriptive  statistics  and  results  of  t  tests  comparing 
participants  who  qualified  experts  and  participants  who  qualified  other  than 
experts.  A  significant  difference  was  found  between  experts  and  non-experts, 
supporting  the  classification.  In  general,  participants  classified  as  experts  reported 
having  a  more  positive  experience  than  non-experts.  With  the  exception  of  the  first 
practice  day,  the  differences  in  scores  were  significant  or  approached  significance. 
These  results  suggest  that  those  participants  who  qualified  expert  were  having 
consistently  better  experiences  throughout  the  practice  period. 
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Table  62 

t  tests  of  Background  Measures  Between  Non-Experts  and  Experts 


Performance  classification 


Sharpshooter  or 

marksman  Expert 


Measure 

n 

M 

SD 

n 

M 

SD 

p  value 

Record-fire  score 

15 

32.73 

4.94 

21 

46.29 

5.22 

<.001 

Frequency  of  shooting  outside  job 

15 

1.67 

0.90 

21 

2.10 

0.21 

.180 

Years  of  shooting  experience  before 

joining  Marines 

15 

1.13 

1.88 

21 

2.90 

1.05 

.137 

Firing  line  experience 

Live-fire  practice  Day  1 

15 

2.36 

0.38 

21 

2.57 

0.09 

.131 

Live-fire  practice  Day  2 

15 

2.33 

0.45 

20 

2.84 

0.11 

.004 

Live-fire  practice  Day  3 

15 

2.64 

0.79 

21 

3.04 

0.10 

.069 

Live-fire  practice  Day  4 

15 

2.82 

0.60 

21 

3.25 

0.11 

.027 

Qualification  day 

14 

2.74 

0.59 

20 

3.48 

0.09 

<.000 

Table  63  shows  descriptive  statistics  of  cognitive  measures  and  results  of  t  tests 
comparing  participants  who  qualified  as  experts  and  participants  who  qualified  as 
other  than  experts.  Significant  differences  were  found  between  experts  and  non¬ 
experts  on  aptitude  (GCT)  and  the  basic  marksmanship  measure.  Participants' 
perceived  level  of  marksmanship  knowledge  is  consistent  with  these  findings,  with 
experts  reporting  a  higher  level  of  understanding  than  non-experts.  There  were  no 
significant  differences  between  experts  and  non-experts  on  any  other  measures  of 
knowledge. 

An  interesting  result  is  the  variation  of  scores  for  each  group.  Participants  in 
the  non-expert  group  had  significantly  higher  variation  across  all  measures, 
compared  to  participants  in  the  expert  group. 
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Table  63 

t  tests  of  Background  Measures  Between  Non-Experts  and  Experts 


Performance  classification 


Sharpshooter  or 

marksman  Expert 


Measure 

ft 

M 

SD 

n 

M 

SD 

p  value 

GCT 

15 

117.80 

7.39 

21 

128.67 

2.33 

.002 

Basic  marksmanship  knowledge 

14 

27.00 

3.11 

20 

29.25 

0.58 

.029 

Shot  group  depiction 

13 

2.08 

0.86 

20 

2.05 

0.22 

.937 

Evaluation  of  shooter  positions 

13 

43.46 

3.31 

20 

41.60 

0.78 

.135 

Knowledge  map 

Pre-classroom  training 

15 

6.33 

6.88 

19 

8.89 

1.70 

.310 

Post-classroom  training 

14 

8.21 

7.49 

14 

9.64 

1.88 

.607 

Live-fire  practice  Day  1 

11 

10.00 

8.09 

18 

8.94 

1.72 

.719 

Live-fire  practice  Day  3 

13 

9.15 

8.48 

16 

11.13 

2.08 

.535 

Qualification  day 

12 

10.00 

8.28 

19 

9.42 

1.79 

.845 

Post-qualification 

14 

12.93 

8.61 

19 

12.63 

1.94 

.922 

Perceived  level  of  marksmanship 

knowledge 

15 

2.77 

0.42 

21 

3.16 

0.07 

.004 

Perceived  utility  of  marksmanship 

knowledge 

15 

1.39 

0.37 

21 

1.39 

0.08 

1.000 

Table  64  shows  descriptive  statistics  of  affective  measures  and  results  of  t  tests 
comparing  participants  who  qualified  experts  and  participants  who  qualified  other 
than  experts.  Significant  differences  were  on  all  worry  measures  (trait  and  state)  and 
two  of  three  state  anxiety  measures.  Consistent  with  firing  line  experience,  experts 
reported  lower  amounts  of  worry  and  anxiety  than  non-experts. 
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Table  64 

t  tests  of  Affective  Measures  Between  Non-Experts  and  Experts 


Performance  classification 


Sharpshooter  or 

marksman  Expert 


Measure 

n 

M 

SD 

n 

M 

SD 

p  value 

Trait  worry,  post-qualification 

14 

3.39 

0.59 

21 

3.75 

0.06 

.048 

State  worry 

Live-fire  practice  Day  1 

15 

1.89 

0.37 

21 

1.54 

0.08 

.007 

Live-fire  practice  Day  3 

15 

1.97 

0.56 

20 

1.46 

0.10 

.005 

Qualification  day 

14 

2.04 

0.65 

19 

1.45 

0.11 

.006 

State  anxiety 

Live-fire  practice  Day  1 

14 

2.11 

0.48 

21 

1.91 

0.11 

.257 

Live-fire  practice  Day  3 

15 

2.31 

0.65 

20 

1.67 

0.11 

.002 

Qualification  day 

14 

2.09 

0.74 

19 

1.64 

0.14 

.068 

Overall,  the  analyses  that  compared  non-experts  to  experts  are  consistent  with 
the  idea  that  there  exists  a  cognitive  component  to  shooting.  Experts  had 
significantly  higher  record-fire  scores,  reported  more  positive  firing  line  experience, 
had  less  worry  and  anxiety  in  general,  and  most  compellingly,  scored  higher  on  the 
cognitive  measure  of  aptitude  (GCT),  basic  marksmanship  knowledge,  and 
perceived  level  of  marksmanship  knowledge.  Interestingly,  there  was  no  difference 
on  the  performance  assessment  measures  (shot  group  depiction,  evaluation  of 
shooter  positions,  and  knowledge  mapping). 

Sensitivity  to  learning.  In  a  second  set  of  analyses  we  examined  whether 
differences  existed  on  test  scores  administered  before  training  started  and  after 
qualification.  The  purpose  of  these  analyses  was  to  gather  information  on  whether 
our  knowledge  measures  were  sensitive  to  presumed  changes  in  learning  over  the 
classroom  and  live-fire  practice  period.  As  Table  65  shows,  there  were  significant 
increases  in  scores  across  all  measures  except  the  evaluation  of  shooter  positions.  In 
this  case,  the  change  was  negative  and  unexpected,  although  the  magnitude  of  the 
change  was  small.  Additional  longitudinal  analyses  were  conducted  on  the 
knowledge  mapping  scores,  and  in  general  show  significant  and  large  change  over 
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the  instructional  period.  An  in-depth  analysis  and  discussion  is  given  in  Appendix 
Z. 


Table  65 

Paired-Sample  Tests  of  Mean  Differences  Between  Measures  Administered  Before  Classroom 
Training  and  After  Qualification  Attempt 


Posttest  Pretest  Differeirce 


n 

M 

SD 

SE 

M 

SD 

SE 

M 

SD 

p  value 

Basic  marksmanship 

29 

28.55 

1.77 

0.51 

22.55 

3.95 

0.73 

6.00 

4.07 

<.001 

knowledge® 

Shot  group  depiction® 

32 

2.06 

0.95 

0.17 

1.69 

1.03 

0.18 

0.38 

1.04 

.050 

Knowledge  map,  post¬ 

31 

12.77 

8.58 

1.54 

7.84 

7.41 

1.33 

4.94 

8.02 

.002 

qualification 

Evaluation  of  shooter 

33 

42.33 

3.48 

0.61 

43.67 

3.58 

0.62 

-1.33 

3.54 

.038 

positions® 


®Post-classroom  training. 


Summary  and  Discussion  of  Study  3 

Study  3  examined  the  prediction  of  record-fire  scores  with  entry-level  2nd  LTs. 
Unlike  Study  1  or  Study  2,  the  sample  was  officers  and  shooting  novices  overall. 

Limitations.  The  biggest  limitation  of  this  study  was  that  one  group  of 
participants  was  dropped  due  to  presumed  weather  effects  on  their  performance.  A 
second  limitation  is  that  the  results  of  this  study  may  generalize  only  to  officers 
(college  educated  in  general)  who  are  novice  shooters.  The  skill-development  theory 
suggests  differential  effects  of  perceptual-motor  and  cognitive  variables  depending 
on  the  experience  of  the  individual,  with  cognitive  variables  having  the  most  impact 
on  performance  during  the  learning  stage  and  perceptual-motor  variables  having 
the  most  impact  on  performance  after  the  learning  phase. 

Reliability  of  measures.  Overall,  our  measures  demonstrated  moderate  to  high 
reliabilities.  The  lowest  reliability  was  with  the  shot  group  depiction  task  (a  =  .23), 
similar  to  prior  studies.  The  basic  marksmanship  knowledge  measure  also  had  a  low 
a  (.64),  given  the  number  of  items.  The  reliability  of  this  measure  has  decreased  over 
studies;  the  reason  behind  this  decline  is  unclear.  The  survey  measures  (e.g., 
affective,  firing  line  experience)  demonstrated  high  reliabilities,  with  a  in  the  .7  to  .9 
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range.  The  knowledge  map  measures  also  demonstrated  high  reliabilities  when 
scores  from  the  different  expert  criterion  maps  were  used  as  items,  a  was  in  the  low 
.9  range. 

Predictive  validity  evidence.  The  best  predictors  of  record-fire  scores  were 
aptitude  (GCT),  basic  marksmanship  knowledge,  firing  line  experience,  and 
affective  measures  (worry  and  anxiety).  Firing  line  experience  was  the  largest 
perceptual-motor  predictor  (r  =  .77),  GCT  the  largest  cognitive  predictor  (r  =  .45), 
and  state  worry  the  largest  affective  (r  =  .64).  Incremental  validity  analyses  indicated 
that  within  each  category  of  variables,  adding  more  variables  contributed  little  to  the 
prediction  of  record-fire  scores.  The  multiple  R  based  on  these  three  variables  was 
.86.  These  results  should  be  interpreted  with  caution— the  sample  size  was  small  for 
these  analyses  (n  =  32). 

Construct  validity  evidence.  Consistent  with  Ackerman's  (1987,  1992)  theory 
of  individual  differences,  particularly  that  performance  during  the  cognitive  phase  is 
influenced  by  aptitude  and  content-relevant  abilities,  we  found  moderate  positive 
correlations  between  record-fire  scores  and  aptitude,  basic  marksmanship 
knowledge,  and  perceived  level  of  marksmanship  knowledge.  However, 
incremental  validity  analyses  showed  that  GCT  was  the  best  predictor  and  the 
addition  of  the  knowledge  variables  contributed  little  to  the  prediction  of  record-fire 
scores. 

The  correlations  among  record-fire  score,  firing  line  experience,  state  anxiety, 
and  state  worry  were  consistent  with  expectations.  Among  the  measures 
administered  on  the  same  day,  the  direction  of  the  relationship  and  the  large 
magnitudes  suggest  that  the  measures  were  working  as  intended.  Firing  line 
experience  was  negatively  associated  with  state  anxiety  and  state  worry. 
Participants  who  reported  more  positive  shooting  experience  also  reported  lower 
anxiety  and  worry.  Similarly,  participants  who  reported  more  anxiety  also  reported 
more  worry.  Trait  worry  was  correlated  significantly  and  moderately  with  firing  line 
experience  (negatively),  state  worry  (positively),  and  state  anxiety  (positively),  a 
result  that  also  is  consistent  with  expectations. 

Sensitivity  of  measures.  Evidence  was  found  for  a  difference  across  nearly  all 
measures  when  participants  who  qualified  as  expert  were  compared  to  others  (i.e., 
sharpshooter  or  marksman).  Expert-qualified  participants  had  more  positive  firing 
line  experiences,  higher  aptitude,  and  more  marksmanship  knowledge,  and 


116 


CSE  Deliverable 


experienced  lower  worry  and  anxiety  while  shooting.  Evidence  was  also  found  for 
the  sensitivity  of  our  knowledge  of  marksmanship  measures  to  training,  with  the 
exception  of  ESP.  Post-training  scores  (after  classroom  and  live-fire  practice)  showed 
significant  gains  compared  to  pre-classroom  training  or  pre-live-fire  training. 

DISCUSSION 

The  set  of  studies  conducted  was  intended  to  investigate  research  questions 
related  to  predicting  record-fire  scores  and  in  general,  gathering  validity  evidence. 
The  type  of  validity  evidence  gathered  was  (a)  construct— evidence  of  knowledge 
and  skill  performance  consistent  with  the  skill-acquisition  model;  (b)  predictive  —  the 
extent  to  which  our  knowledge,  perceptual-motor,  and  affective  measures  predicted 
record-fire  performance;  (c)  evidence  of  a  relationship  among  knowledge  measures; 
and  (d)  evidence  of  the  sensitivity  of  knowledge  measures  —  to  instructional  effects 
and  knowledge  differences.  Table  66  to  Table  68  summarize  the  empirical  evidence 
regarding  the  predictability  of  record-fire  scores,  and  group  differences  with  respect 
to  perceptual-motor,  cognitive,  and  affective  variables  for  shooters  of  different 
backgrounds  (i.e.,  performance  and  experience). 
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Firing  line  experience  (pre-qual.): 
Low  <  Others 
SLR  <  Coaches 

Firing  line  experience  (post-qual.): 
Low  <  Others 


Table  68 


Pretest-Posttest  Differences  on  Cognitive  Measures 


Study 

Measure 

Direction  of  Group  Difference 

Study  2 

Basic  marksmanship  knowledge 

Knowledge  map 

Posttest  >  pretest 

Posttest  >  pretest 

Study  3 

Basic  marksmanship  knowledge 

Posttest  >  pretest 

(n  =  32) 


Knowledge  map 
Shot  group  depiction 
Evaluation  of  shooter  positions 


Posttest  >  pretest 
Posttest  >  pretest 
Posttest  <  pretest 
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In  the  following  sections  we  summarize  the  empirical  evidence  with  respect  to 
the  broad  research  questions  guiding  the  studies.  Table  5  contains  a  more  detailed 
mapping  of  the  evidence  and  specific  expectations. 

To  What  Extent  Can  Record-Fire  Scores  Be  Predicted? 

Evidence  of  predictive  validity  was  obtained  via  multiple  regression  analyses 
and  bivariate  correlations.  Combining  perceptual-motor,  cognitive,  and  affective 
variables  to  predict  record-fire  scores  for  Pilot  Studies  1  and  2,  and  Studies  1,  2,  and 
3,  resulted  in  the  following  Rs:  .62,  .53,  .56,  .52,  and  .86  (note  small  sample  size  for 
Study  3). 

Bivariate  correlations  between  various  measures  and  record-fire  scores  were 
obtained  in  the  .2  to  .8  range.  Perceptual-motor  measures— intended  to  reflect 
experience— were  consistently  a  good  predictor  of  performance.  The  most  recent 
record-fire  score  predicted  record-fire  score  at  the  .3  to  .4  range.  The  best  single 
predictor  of  record-fire  score  was  the  firing  line  experience  survey,  which  yielded 
correlation  coefficients  from  .6  to  .8.  Cognitive  measures  (aptitude  and  knowledge 
related  to  marksmanship)  in  less  experienced  samples  related  to  record-fire  score  in 
the  .2  to  .4  range.  No  relationships  between  record-fire  score  and  knowledge 
measures  were  found  in  the  more  experienced  sample.  Affective  measures  (worry, 
anxiety)  predicted  record-fire  scores  in  the  -.3  to  -.6  range  and  in  general,  for  the 
affective  and  firing  line  experience  measures,  state  measures  had  coefficients  of 
higher  magnitude  than  the  trait  versions. 

Consistent  with  our  expectations,  there  appeared  to  be  a  general  difference 
between  samples  with  respect  to  the  sensitivity  of  our  perceptual-motor  and 
cognitive  measures.  Perceptual-motor  measures  were  better  predictors  of 
performance  in  the  more  experienced  sample,  and  cognitive  measures  were  better 
predictors  of  performance  in  the  less  experienced  sample.  Worry  appeared  to  be  an 
important  factor  as  well,  showing  up  as  moderate  to  strong  negative  predictors  of 
shooting  performance  in  Studies  2  and  3  (unfortunately,  we  did  not  use  the  measure 
in  earlier  studies). 

The  magnitude  of  the  predictive  validity  of  our  measures  was  similar  to  prior 
work.  The  highest  reported  correlation  was  in  the  .8  to  .9  range,  and  this  result  was 
part  of  a  test-retest  analysis  of  Army  soldiers'  qualification  scores  on  the  Ml  rifle 
(McGuigan  &  MacCaslin,  1955).  Prior  shooting  experience  and  aptitude  jointly 
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predicted  record-fire  scores  in  the  .7  range  (MacCaslin  &  McGuigan,  1956)  as  did 
rifle  steadiness  (Humphreys  et  ah,  1936).  Interestingly,  performance  on  an  M16A2 
rifle  simulator  related  to  record-fire  performance  in  the  .5  to  .7  range  (Hagman,  1998; 
Schendel,  et  al.,  1985;  Smith  &  Hagman,  2000;  Torre  et  al.,  1987). 

The  particular  finding  that  simple  measures  of  firing  line  experience  and  worry 
can  predict  record-fire  scores  as  well  as  performance  on  a  rifle  simulator  is 
remarkable  and  of  high  practical  utility.  That  is,  these  measures  —  quick  to  complete 
and  simple  to  administer  — could  be  used  as  a  screening  device.  Marines  identified 
as  at-risk  for  poor  performance  could  then  be  routed  for  early  remediation  training. 
Assessment  used  in  conjunction  with  Web-based  delivery  could  be  used  to  identify 
knowledge  gaps  and  provide  online  remediation  —  before  the  shooter  ever  reaches 
the  firing  line. 

What  Is  the  Role  of  Cognitive  Variables  in  Rifle  Marksmanship  Performance? 

Aptitude.  The  clearest  example  that  aptitude  matters  is  shown  in  Study  3,  the 
only  study  where  aptitude  was  available.  GCT  scores  were  a  very  strong  predictor 
of  shooting  scores  and  expert-qualified  participants  had  higher  GCT  scores  than 
lower  performing  participants.  This  finding  is  consistent  with  Ackerman's  model  of 
individual  differences  (1987,  1992),  which  specifies  aptitude  and  content-related 
abilities  as  important  predictors  of  skill  development  for  trainees  learning  a  new 
skill. 

Knowledge.  Analyses  of  group  differences  by  performance  and  by  experience 
clearly  show  differences  in  knowledge.  Basic  marksmanship  knowledge  consistently 
showed  correlations  with  record-fire  scores  (rs  in  the  .2  to  .3  range)  in  less 
experienced  samples.  Tests  of  group  differences  also  showed  coaches  and  high 
performers,  compared  to  others,  consistently  scored  higher  on  the  basic 
marksmanship  measure. 

However,  the  unique  contribution  of  knowledge  toward  the  prediction  of 
record-fire  score  is  not  nearly  as  strong  as  perceptual-motor  measures  (e.g.,  most 
recent  record-fire  score).  This  is  consistent  with  expectations.  We  speculate  the  role 
of  knowledge  may  be  a  second-order  effect;  that  is,  knowledge  may  be  important  to 
shooting  well,  but  specific  circumstances  at  the  time  of  qualification  may  be  the  best 
predictor  of  performance.  Given  the  extreme  sensitivity  of  shooting  to  minute 
movements,  factors  that  unsettle  the  shooter  (e.g.,  poor  weather,  intimidating  coach. 
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an  off  day)  may  result  in  episodic  performance.  For  example,  the  strongest 
predictors  of  record-fire  scores  were  the  measures  that  asked  about  participants' 
state  at  the  time  of  qualification:  how  was  their  firing  experience,  and  how  worried 
or  how  anxious  were  shooters  when  they  attempted  to  qualify.  Trait  measures  of 
worry  and  firing  line  experience  show  a  lower  (but  still  significant)  relationship  with 
record-fire  score,  compared  to  state  versions  of  the  measure.  Thus,  it  may  be 
necessary  to  examine  performance  over  time  to  get  a  more  stable  estimate  of  where 
trainees  are  in  their  skill  development. 

In  Study  1  we  had  a  sample  distribution  that  allowed  us  to  conduct  an 
exploratory  analysis  using  such  an  approach  (i.e.,  categorize  shooters  in  terms  of 
their  consistency  and  level  of  performance  over  time).  In  this  case,  the  correlation  of 
basic  marksmanship  knowledge  to  record-fire  scores  was  nearly  twice  as  large  for 
participants  classified  as  learners  (i.e.,  r  =  .55,  p  <  .05;  participants  with  high 
standard  deviations  of  shooting  scores  over  time)  compared  to  participants 
classified  as  high  performers  (r  =  .29,  n.s.). 

Overall,  the  results  of  these  studies  suggest  that  there  exist  differences  in 
knowledge  of  rifle  marksmanship  between  participants'  pre-classroom  training  and 
post-classroom  training,  between  more  experienced  participants  and  less 
experienced  participants,  between  high  performers  and  low  performers,  and 
between  higher  aptitude  and  lower  aptitude.  What  remains  unclear  is  how 
knowledge  of  rifle  marksmanship  operates  to  influence  shooting  performance.  That 
is,  how  much  knowledge  and  what  kind  of  knowledge  does  a  shooter  need  in  order 
to  practice  effectively  and  rapidly  transition  from  a  cognitive  (i.e.,  learning)  phase  to 
an  associative  (i.e.,  practice)  phase. 

What  Is  the  Overall  Quality  of  the  Assessment  Measures? 

Overall,  our  cognitive  measures  predicted  record-fire  scores  best  with  less 
experienced  samples.  Of  all  the  knowledge  measures,  the  basic  marksmanship 
knowledge  was  the  most  consistent  measure,  yielding  low  to  moderate  correlations 
with  record-fire  score. 

Analyses  of  performance  on  our  knowledge  measures  by  experience  and  by 
performance  were  consistent  with  expectations.  Low  performers  scored  lower  on  the 
basic  marksmanship  knowledge  measure,  and  they  perceived  their  level  of 
marksmanship  knowledge  to  be  lower  than  high  performers.  This  finding  was 
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observed  in  every  study.  In  addition,  in  Study  2,  participants  who  had  just 
completed  the  Marksmanship  Coaches  Course,  compared  to  the  main  SLR  sample, 
scored  higher  on  the  knowledge  map  and  shot  group  depiction  tasks  (the  coaches 
were  not  available  to  take  the  basic  marksmanship  knowledge  task).  Finally,  for 
participants  in  Studies  2  and  3  who  were  available  for  pre-  and  post-testing,  higher 
scores  on  the  posttest  were  observed  for  the  basic  marksmanship  knowledge  and 
knowledge  mapping  measures. 

Low  to  moderate  intercorrelations  among  the  knowledge  measures  were 
observed  in  Studies  1  and  2.  In  particular,  the  correlations  between  the  basic 
marksmanship  knowledge  and  shot  group  depiction  and  knowledge,  were  higher  in 
Study  1  (the  less  experienced  sample)  than  in  Study  2.  However,  these  relationships 
were  not  observed  in  Study  3,  the  least  experienced  sample.  In  general,  the 
correlations  among  the  knowledge  measures  in  Study  1  were  most  consistent  with 
expectations  (related  but  not  complete  overlap).  Study  2  showed  no  correlations 
among  the  performance  measures  (only  the  basic  marksmanship  knowledge 
measure  related  with  other  measures). 

Study  3  provided  an  interesting  sample:  A  measure  of  aptitude  was  available 
and  was  shown  to  relate  to  knowledge  mapping  (pretest)  and  the  basic 
marksmanship  knowledge  score.  The  pretest  knowledge  map  scores  were  related 
significantly  to  the  posttest  knowledge  map  scores,  but  the  post-classroom  training 
knowledge  map  scores  did  not  relate  to  the  posttest  knowledge  map  scores.  This  is 
interesting  because  (as  discussed  in  Delacruz,  Chung,  &  Bewley,  2003),  many 
participants'  initial  maps  reflect  higher  conceptual  relations,  only  to  be  changed  after 
instruction  to  reflect  hierarchical  relations  (as  taught  in  the  training).  In  any  case,  the 
mean  knowledge  map  scores  increased  over  occasion.  While  posttest  map  scores 
were  significantly  higher  than  pretest  scores,  the  relative  ranking  of  participants 
changed  over  time. 

Overall,  we  have  gathered  evidence  that  in  general  suggests  that  our 
knowledge  measures  are  sensitive  to  instruction,  and  knowledge  measures  can 
predict  record-fire  scores  moderately  in  less  experienced  samples,  and  when 
combined  with  other  variables  within  the  stages-of-  skill-processing  framework,  can 
predict  record-fire  scores  as  well  as  scores  from  a  rifle  simulator. 

While  we  expected  that  perceptual-motor  variables  would  be  the  most 
important  predictor  of  performance,  the  low  predictive  validity  of  our  performance 
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measures  (particularly  the  knowledge  mapping  and  evaluation  of  shooter  positions 
tasks)  was  unexpected.  Our  review  of  tasks  and  measures  with  respect  to  the  SLR 
curriculum  and  training  suggests  that  the  performance  tasks  may  have  demanded 
too  much  of  participants.  From  a  practical  stance,  entry-level  and  sustainment-level 
marksmanship  training  is  limited  in  scope  and  focuses  on  communicating  to  large 
groups  of  Marines  a  basic  level  of  knowledge:  safety,  weapons  handling,  positions, 
and  the  bare  minimum  of  declarative  and  procedural  knowledge  of  the 
fundamentals  of  marksmanship.  Causal  relations  are  covered  briefly  if  at  all  (e.g., 
how  movement  in  one  part  of  the  body  affects  movement  in  another  part),  and  the 
task  of  identifying  poor  shooting  positions  is  considered  a  coaching  rather  than  a 
shooter  function. 

The  idea  that  our  performance  tasks  targeted  too  deep  a  level  of  knowledge  is 
consistent  with  our  findings  in  two  ways.  First,  of  all  the  knowledge-based 
measures,  the  basic  marksmanship  knowledge  test  was  consistently  the  best 
predictor  of  record-fire  score  (although  low  to  moderate).  Second,  in  Study  2, 
participants  who  had  just  completed  the  Marksmanship  Coaches  Course  performed 
significantly  higher  on  the  knowledge  map  and  shot  group  depiction  tasks  than  the 
main  SLR  sample. 


Conclusion 

In  the  series  of  studies  on  the  prediction  of  rifle  marksmanship  performance  we 
found  broad  evidence  for  the  idea  that  rifle  marksmanship  —  shooting  performance 
and  knowledge  of  rifle  marksmanship— was  consistent  with  the  stages  of  skill 
acquisition  framework.  We  found  evidence  of  performance  and  knowledge 
differences  by  participants  presumably  in  different  stages  of  development  (i.e., 
learning  vs.  practice  stages),  evidence  that  our  measures  were  sensitive  to 
instruction,  and  evidence  that  our  measures  differentiated  between  those  who 
presumably  know  more  and  those  who  know  less.  We  found  the  relative 
contribution  of  perceptual-motor  measures  to  be  good  predictors  of  shooting 
performance  in  general,  although  knowledge  was  also  predictive  (but  lower  in 
magnitude)  of  shooting  performance.  We  also  found  evidence  that  state  worry  was 
the  highest  predictor  of  shooting  performance. 

Our  findings  are  provocative  for  two  reasons.  First,  the  evidence  suggests  that 
record-fire  scores  can  be  predicted  with  a  variety  of  measures,  as  well  as  other  forms 
of  performance.  This  is  particularly  promising  if  screening  or  remediation  will  be  via 
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ADL:  ADL  rules  out  the  observation  of  shooting  performance;  thus,  being  able  to 
predict  shooting  performance  via  background  and  knowledge  measures  is  essential. 
Additionally,  using  knowledge  measures  also  yields  diagnostic  information  (i.e., 
what  participants  do  not  know). 

Second,  the  general  finding  that  less  experienced  shooters  and  lower 
performing  shooters  had  less  rifle  marksmanship  knowledge  is  encouraging  as  well. 
This  finding  is  important  for  practical  reasons.  Bolstering  knowledge  is  feasible  in  an 
ADL  context.  If  screening  is  combined  with  diagnosis  of  knowledge  gaps,  then 
training  replicating  the  content  of  current  classroom  instruction  can  be  done  using 
existing  technology  platforms  and  standards  (e.g.,  SCORM). 

Rifle  marksmanship  is  a  complex  psychomotor  skill  sensitive  to  variations  in 
the  individual,  equipment,  and  environment.  It  is  unlikely  that  variation  in  the 
equipment  and  environment  can  be  reduced  much,  thus  leaving  the  individual  as 
the  only  area  for  improvement.  Prior  research  in  other  sports  (e.g.,  tennis)  suggests 
that  improving  the  motor  dimension  involves  physical  practice  of  the  skill  and  takes 
much  longer  to  gain  competency  than  improving  the  knowledge  dimension 
associated  with  the  skill.  Given  that  we  have  found  a  cognitive  component  to  rifle 
marksmanship  performance,  it  may  be  that  improving  a  Marine's  knowledge  of  rifle 
marksmanship  will  have  the  most  cost-effective  payoff.  What  is  unclear  is  how  large 
the  effect  would  be  on  overall  shooting  performance  and  how  lasting  the 
improvement  would  be. 


Next  Steps 

A  clear  next  step  is  to  develop  screening  measures  that  can  identify  individual 
Marines  likely  to  fail  qualification.  Given  the  large  contribution  of  perceptual-motor 
variables  to  the  predictability  of  record-fire  score  it  may  be  that  an  efficient  method 
to  identify  potential  UNQs  would  be  to  develop  a  prediction  model  using 
individual-level  data  from  the  Marine  Corps  Central  Master  File  of  the  Marine 
Corps  Total  Force  System  (MCTFS)  database.  This  research  would  analyze  record- 
fire  performance  with  respect  to  the  phases-of-skill-learning  model  that  we  have 
adopted.  Potential  marksmanship  data  to  be  examined  include  rifle,  pistol,  and 
field-firing  performance.  Longitudinal  information  is  available  from  the  MCTFS  that 
includes  marksmanship  qualification  history,  test  scores,  billet  history,  and  military 
and  civilian  education  history.  Potential  approaches  may  be  to  develop  a  prediction 
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model  based  on  experience  and  capacity-to-leam  variables,  and  test  on  a  cross- 
validation  sample. 

Such  a  study  would  have  theoretical  and  practical  implications.  With  respect  to 
theory,  a  longitudinal  analysis  should  inform  us  on  how  rifle  marksmanship  follows 
the  phases-of-skill-learning  model.  Presumably,  aptitude  variables  should  be  more 
related  to  shooting  for  less  experienced  personnel,  and  less  related  to  performance 
for  more  experienced  personnel.  High-aptitude  personnel  should  perform  higher 
given  the  same  level  of  experience,  but  this  difference  should  diminish  over  time.  In 
general,  shooting  scores  should  follow  the  power  law:  rapid  increase  in  scores 
initially  with  diminishing  increases  over  time.  The  practical  outcome  of  this  research 
would  be  a  way  to  quickly  identify  Marines  at  risk  for  failing  to  qualify.  Once  a 
Marine  is  identified,  then  diagnostic  assessments  could  be  administered,  and 
remediation  can  occur  via  ADL-based  instruction  prior  to  the  Marine  reaching  the 
firing  line. 
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Audible  Pop  or  Reduced  Recoil 

An  audible  pop  occurs  when  only  a  portion  of  the  propellant  is  Declarative,  factual 

ignited.  It  is  normally  identifiable  by  reduced  recoil  and  is  sometimes  (knowledge  of 
accompanied  by  excessive  smoke  escaping  from  the  chamber  area.  To  terminology) 
clear  the  rifle  in  a  combat  environment: 
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Aiming  •  Sight  alignment  is  the  relationship  between  the  front  sight  post  and  Declarative,  factual  (MCRP)  3-OlA 

Sight  alignment  rear  sight  aperture  and  the  aiming  eye.  (knowledge  of  Chapter  4 

terminology)  SecHon4001 
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APPENDIX  B 


KEY  RIFLE  MARKSMANSHIP  FACTS 
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Topic 

Description 

Weapons  handling  A  Marine  carries  the  rifle  at  Tactical  Carry  when  there  is  no  immediate  threat 
present. 

Weapons  handling  Definition  of  remedial  action 


Remedial  action  is  investigating  the  cause  of  the  stoppage,  clearing  the 
stoppage,  and  returning  the  weapon  to  operation. 


Weapons  handling: 
Weapon  condition 


Weapons  handling: 
Remedial  Action 


Different  weapons  conditions  (could  be  graphic) 

CONDITION  1:  Magazine  inserted,  round  in  chamber,  bold  forward,  safety 
on,  ejection  port  cover  closed 

CONDITION  2:  Not  Applicable 

CONDITION  3:  Magazine  inserted,  chamber  empty,  bolt  forward,  safety  on, 
ejection  port  cover  closed 

CONDITION  4:  Magazine  removed,  chamber  empty,  bold  forward,  safety  on, 
ejection  port  cover  closed 

Three  indicators  of  remedial  action  and  their  corresponding  actions 

(1) Bolt  is  forward  or  ejection  port  cover  is  closed.  ACTION  Tap,  rack,  bang 

(2) Bolt  is  locked  to  the  rear.  ACTION  Conduct  a  dry  reload, 

(3) Brass  is  obstructing  chamber  area  (usually  indicates  double  feed  or  failure 
to  eject)  ACTION  Remove  magazine.  Lock  bolt  to  rear.  Shake  rounds  out. 
Conduct  a  reload. 


Weapons  handling: 
Rifle  transfer 


Weapons  handling: 
Safety  Rules 


Fundamentals  of 
Marksmanship 


Fundamentals  of 
Marksmanship: 

Aiming 

Fundamentals  of 
Marksmanship: 

Aiming 

Fundamentals  of 

Marksmanship: 

Aiming 


Two  procedures  to  use  to  transfer  a  rifle  from  one  Marine  to  another 
Clear  transfer 

Condition  Unknown  Transfer 

Four  safety  rules  that  most  strongly  enforce  muzzle  awareness. 

Treat  every  weapon  as  if  it  were  loaded. 

Never  point  a  weapon  at  anything  you  do  not  intend  to  shoot. 

Keep  your  finger  straight  and  off  the  trigger  until  you  are  ready  to  fire. 

Keep  the  weapon  on  safe  until  you  intend  to  fire. 

The  three  fundamentals  of  marksmanship 
Aim 

Breathing  Control 
Trigger  control 

Obtaining  a  natural  point  of  aim 

Align  sights,  breathe  in,  close  your  eyes,  exhale,  and  open  your  eyes. 

Achieving  a  correct  sight  picture. 

Place  the  tip  of  the  front  sight  post  at  the  center  of  the  target  while  maintaining 
sight  alignment. 

Definition  of  center  mass. 

The  correct  aiming  point  so  that  point  of  aim/ point  of  impact  is  achieved. 
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Topic 

Description 

Fundamentals  of  Where  the  shooter's  main  point  of  focus  should  be  when  he/ she  fires  a  shot 

Marksmanship:  Center  mass  or  center  of  target 

Aiming 


Fundamentals  of 
Marksmanship: 
Breath  control 


Fundamentals  of 
Marksmanship: 
Trigger  control 


The  technique  for  breath  control  is  as  follows: 

Breathe  naturally  until  the  sight  picture  begins  to  settle. 

Take  a  slightly  deeper  breath 

Exhale  and  extend  your  natural  respiratory  pause 

Stop  breathing  at  your  point  of  NRP  before  firing  a  long-range  shot 

Definition  of  uninterrupted  Trigger  Control.  Definition  of  interrupted  Trigger 
Control 

Uninterrupted  Trigger  Control:  After  the  initial  slack  of  the  trigger  is  taken  up, 
the  trigger  is  pulled  with  a  single,  smooth  motion  straight  to  the  rear  with  no 
interruption 

Interrupted  Trigger  Control:  After  the  initial  slack  is  taken  up,  the  trigger  is 
moved  to  the  read  unless  an  error  is  detected  in  the  aiming  process.  When  this 
occurs,  rearward  motion  is  topped  until  sight  picture  is  achieved.  Then  the 
rearward  motion  continues  until  the  shot  breaks. 


Fundamentals  of 
Marksmanship: 
Trigger  Control 

Fundamentals  of 
Marksmanship: 

Trigger  Control 

Fundamentals  of 
Marksmanship: 

Trigger  Control 


Use  of  interrupted  trigger  control 

In  extremely  windy  conditions  when  the  weapon  will  not  settle,  forcing  the 
Marine  to  pause  until  the  sights  return  to  his  aiming  point. 

Uninterrupted  trigger  control  is  the  preferred  method  in  a  combat 
environment 


Definition  of  a  follow-through 

The  continued  applications  of  the  fundamentals  until  the  round  has  exited  the 
barrel. 


Rifle  firing  positions:  The  most  stable  position  is  the  prone  position.  The  least  stable  position  is  the 
Stability  standing  position 


Rifle  firing  positions:  The  least  mobile  of  the  shooting  positions 
Mobility  Prone 


Rifle  firing  positions:  The  best  firing  position  that  normally  provides  the  best  field  of  view  is 

Observation  of  the  standing 

enemy 


Rifle  firing  positions:  The  two  basic  types  of  rifle  sling  adjustments 
Types  and  uses  of  Hasty  Sling  and  Loop  Sling 
Rifle  web  sling 


Rifle  firing  positions:  If  body  alignment  is  correct,  the  weapon's  recoil  is  absorbed  by  the  whole 
Types  and  uses  of  body 
Rifle  web  sling 


i 


i 


i 


« 
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Topic 


Description 


Rifle  firing  positions: 
Types  and  uses  of 
Rifle  web  sling 

Rifle  firing  positions: 
Elements  of  a  good 
shooting  position 


Rifle  firing  positions: 
Elements  of  a  good 
shooting  position 


Rifle  firing  positions: 
Elements  of  a  good 
shooting  position 

Effects  of  weather: 
On  marines 


Effects  of  weather: 
On  the  bullet 
Zeroing: 

Elements  of  zeroing 


Zeroing: 

Types  of  zeros 
Battlesight  Zero 
Zeroing: 
Battlesight  Zero 


The  rifle  sling  provides  maximum  stability  for  the  weapon  and  helps  stabilize 
the  front  sight  and  reduce  the  effects  of  the  rifle's  recoil. 

Three  elements  of  a  good  shooting  position. 

Bone  support 
Muscular  Relaxation 
Natural  Point  of  Aim 

Seven  factors  that  are  common  to  all  shooting  positions  as  they  apply  with  the 
Hasty  Sling 
Left  hand 

Rifle  Butt  in  the  Pocket  of  the  Shoulder 

Grip  of  the  Right  Hand 

Right  Elbow 

Stock  Weld 

Breathing 

Muscular  Tension 

In  the  kneeling  position,  a  right-handed  shooter  should  have  his/her  right 
elbow  supported. 

Weather  factors  that  affect  the  shooter 
Wind 

Temperature 

Precipitation 

Wind  affect  the  bullet's  trajectory  laterally 

Five  basic  elements  involved  in  zeroing  the  rifle 

Line  of  sight 

Point  of  aim 

Centerline  of  the  bore 

Trajectory 

Range 

A  BZO  is  the  elevation  and  windage  setting  that  is  used  in  combat  to  engage 
point  targets  from  0-300  yards/meters  under  now  wind  conditions. 


Factors  that  affect  the  accuracy  of  a  BZO 

Forward  hand,  grip,  right  elbow,  stock  weld,  rifle  butt  in  the  pocket  of  the 
shoulder,  relaxation,  breathing 

Sling  tension,  trigger  control,  slight  picture 
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Topic 

Description 

Zeroing:  Some  factors  that  cause  a  BZO  to  be  reconfirmed. 

Reconfirmation  of  Rifle  maintenance,  temperature,  climate,  ground  elevation,  and  uniform 
BZO 


Zeroing:  Front  sight  is  used  to  adjust  for  elevation 

Sight  alignment 


Zeroing: 

Sight  alignment 


Zeroing: 

Sight  alignment 


The  direction  to  rotate  the  front  sight  post  in  order  to  move  the  strike  of  the 
round  up.  And  down. 

To  move  it  up,  move  it  right  (or  clockwise) 

To  move  it  down,  move  it  left  (or  counterclockwise) 

During  the  zeroing  process,  all  elevation  adjustments  should  be  made  on  the 
front  sight  post 


Zeroing: 

Sight  alignment 

Zeroing: 

Sight  alignment 

Engagement 

techniques: 

Target  indicators 


The  direction  to  turn  the  rear  sight  windage  knob  in  order  to  move  the  strike 
of  the  round  to  the  right. 

To  the  right  or  clockwise 

The  rear  sight  elevation  should  be  set  on  8/3  -  2  when  firing  at  the  200-yd  line 

Three  target  indicators 

Movement 

Sound 

Improper  Camouflage 
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APPENDIX  C 

KEY  RIFLE  MARKSMANSHIP  CAUSE-EFFECT  RELATIONS 
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Fundamental  Impacts 


Explanation 


Stock  weld 

Sight  picture  and 
alignment 

Maintains  an  erect  head  position,  which  places  the  eye 
in  its  natural  forward  position,  enabling  the  aiming  eye 
to  look  straight  through  the  rear  sight  picture. 

If  the  position  of  the  Marine's  head  causes  him  to  look 
across  the  bridge  of  his  nose  or  out  from  under  his 
eyebrow,  the  eye  will  be  strained. 

Varying  the  stock 
weld  placement 

Perception  of  the  rear 
sight  aperture 

Changes  eye  relief 

Eye  relief  (eye  is 
too  close  to  the 
rear  sight 
aperture) 

Sight  picture  and 
alignment 

Makes  it  difficult  to  line  up  the  front  sight  post  if  the  eye 
is  too  close  to  the  rear  sight  aperture. 

Eye  relief  (eye  is 
too  far  from  the 
rear  sight 
aperture) 

Sight  picture  and 
alignment 

Makes  it  difficult  to  acquire  the  target  and  to  maintain  a 
precise  aiming  point. 

Focus  of  eye  on 
front  sight  post 

Sight  picture  and 
alignment 

Enables  detection  of  minute  errors  in  sight  alignment 
and  sight  picture. 

Marine's  focus  should  shift  repeatedly  from  the  front 
sight  post  to  the  target  until  correct  sight  alignment  and 
sight  picture  are  obtained. 

Focus  of  eye  on 
front  sight  post 
when  shot  is  fired 

Sight  alignment 

Distorts  the  image  when  staring  at  the  front  sight  post 
for  longer  than  a  few  seconds,  making  it  difficult  to 
detect  minute  errors  in  sight  alignment. 

Increasing 
distance  to  the 
target 

Ability  to  aim  at 
center  of  mass  and 
maintain  a  center 
mass  sight  picture 

Front  sight  post  covers  more  of  the  target  making  it 
difficult  to  establish  a  center  of  mass. 

Looking  at  the 
target 

Impact  of  shots 

Lowers  the  tip  of  the  front  sight  post,  which  causes 
shots  to  impact  low  or  miss  the  target  completely. 

Breath  control 

Sight  picture 

Allows  the  Marine  to  fire  the  rifle  at  the  moment  of  least 
movement  since  breathing  causes  the  body  to  move 
which  transfers  to  the  rifle,  making  it  impossible  to 
maintain  proper  sight  picture 

Trigger  control 
(firm  grip) 

Sights 

If  it  is  firm  enough,  it  should  allow  manipulation  of  the 
trigger  without  disturbing  the  sights. 

Trigger  control 

(finger 

placement) 

Sight  alignment 

If  the  trigger  finger  contacts  the  trigger  naturally,  it 
should  allow  the  trigger  to  be  pulled  straight  to  the  rear 
without  disturbing  sight  alignment 

Follow-through 

Impact  of  the  round 

Keeps  rifle  as  still  as  possible  until  round  exits  the 
barrel. 
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Fundamental 

Impacts 

Explanation 

Stable  firing 
position 

Sights 

Front  sight  can  be  held  steady  and  the  rifle  sights 
should  recover  after  recoil  to  the  same  position  on  the 
target. 

Mobile  firing 
position 

Engagement 

Standing  allows  the  most  lateral  movement  for 
engagement  of  widely  dispersed  targets. 

Rifle  sling 

Sight  alignment  and 
rifle  recoil 

Provides  maximum  stability  for  the  weapon,  which 
reduces  the  effects  of  rifle  recoil  and  offers  resistance 
against  the  sling,  which  enables  the  rifle  sights  to  be 
held  steady. 

Varying  of  sling 

BZO 

Changes  the  strike  of  the  bullet. 

tension 

Controlled 
muscle  tension  of 

Sights 

Offers  resistance  against  the  sling,  keeping  sights 
steady. 

hasty  sling 

Forward  hand 
placement 

Stability  of  rifle 

A  straight  and  locked  forward  hand's  wrist  creates 
resistance  on  the  sling  close  to  the  muzzle,  stabilizing 
the  rifle. 

If  the  rifle  rests  across  the  palm  of  the  hand,  the  only 
resistance  created  is  where  the  sling  meets  the  triceps. 

Resistance  is  further  from  the  muzzle,  making 
stabilizing  the  rifle  more  difficult. 

Excessively 
tightened  loop 
sling 

Rifle  sights 

Restricting  blood  flow  causes  an  excessive  pulse  beat  to 
be  transmitted  through  the  rifle  sling  to  the  rifle,  and 
causes  a  rhythmic  movement  of  the  rifle  sights. 

Tension  on  the 
rifle  sling 

Rifle  recoil 

Causes  the  rifle  butt  to  be  forced  rearward  into  the 
pocket  of  the  shoulder,  keeping  butt  plate  in  the 
shoulder  pocket  during  recoil. 

Bone  support 

Weapon  movement 

Provides  a  stable  foundation  to  support  the  rifle's 
weight  because  muscles  fatigue  whereas  bones  do  not. 

Muscular 

relaxation 

Accuracy  of  aim 

Creates  a  minimum  arc  of  movement  and  consistency  in 
resistance  to  recoil  by  permitting  the  use  of  maximum 
bone  support.  Tense  muscles  cause  excessive  movement 
of  the  rifle,  which  disturbs  aim. 

Velocity  of  wind 

Deflection  of  the 
bullet 

The  greater  the  velocity  of  the  wind,  the  more  the  bullet 
will  be  deflected. 

Rifle  chamber's 
pressure  (cold 
weather) 

Point  of  aim 

In  cold  weather,  the  rifle  chamber's  pressure  decreases, 
which  causes  the  bullet  to  exit  the  muzzle  at  a  lower 
velocity,  which  impacts  the  target  below  the  point  of 
aim. 

Rifle  chamber's 
pressure  (extreme 
heat) 

Point  of  aim 

In  extreme  heat,  the  rifle  chamber's  pressure  increases, 
which  causes  the  bullet  to  exit  the  muzzle  at  a  higher 
velocity,  which  impacts  the  target  above  the  point  of 
aim. 
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Fundamental 

Impacts 

Explanation 

Rapid  fatigue 

Sight  alignment 

Rapid  fatigue  causes  muscle  cramps,  heat  exhaustion, 
heat  stroke,  blurred  vision,  and  reduced  concentration, 
which  results  in  inaccurate  shooting. 

Sweat  running 
into  the  eyes 

Sights 

Sweat  running  into  the  eyes  can  cause  irritation,  which 
makes  it  difficult  to  see  the  sights. 

Heat  waves  or 
mirages 

Sights 

Distorts  the  target  shape  or  the  appearance  of  the  front 
sight  post,  reducing  the  Marine's  ability  to  see  the  sight 
clearly. 

Extreme  cold 

Trigger  control 

If  hands  are  numb,  the  Marine  will  have  difficulty 
holding  a  rifle  and  executing  effective  trigger  control. 

Precipitation 

Sight  alignment  and 
picture 

When  it  collects  on  rear  sight  aperture,  it  can  make  it 
difficult  to  establish  sight  alignment  and  sight  picture. 

Bright  light 

Appearance  of  a 
target 

Makes  a  target  appear  smaller  and  farther  away. 

Overcast 

Appearance  of  a 
target 

Makes  the  target  appear  larger  and  closer. 

Haze 

Sight  picture 

Makes  a  target  appear  indistinct. 

Temperature 

BZO 

Causes  chamber  pressure  to  increase  or  decrease, 
causing  the  shots  to  impact  the  target  high  or  low 
^respectively). 

Climate 

BZO 

Changes  air  density,  moisture  content,  temperature  or 
barometric  pressure. 

Ammunition 

BZO 

Inconsistencies  in  the  production  of  ammunition  affects 
BZO. 

Ground  elevation 

BZO 

Creates  changes  in  air  density,  moisture  content, 
temperature,  or  barometric  pressure. 

Uniform 

BZO 

Changes  eye  relief,  placement  of  the  rifle  in  the  shoulder 
pocket,  and  the  way  the  rifle  is  supported  on  the 
handguard. 
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APPENDIX  D 

SHOT-TO-SHOT  EXPLANATION  TASK  EXAMPLE  (PILOT  STUDY  1) 
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U  Increased  wind  vefocSy. 

U  Decreased  wind  velocfr/. 

□  Wind  is  blowing  Irom  the  teft  but  inadequate  sight 
adjustrnerits  v/ere  made  to  eorripehsate, 

□  Wind  is  blowing  from  lha  right  but  inadequate  stglil 
adjustments  v/ere  made  to  compensate. 

□  Not  executing  proper  follow-through. 

□  Moved  forward  hand  forward  on  the  handguand,  while  firing. 

Moved  forward  hand  backvjard  on  the  handguard,  v;hile  tiring, 

■^Improper  breathing  control. 

Did  not  attain  natural  point  H  aim. 

Moved  the  buttstock  placertiert  tower  on  the  shouitter.  while  firing. 

□  Moved  the  buttstock  placement  higher  on  the  shoulder,  while  firing, 

□  Mcr/ed  forward  elbow  forward  on  the  knee,  while  firing. 

□  Moved  forward  elbovr  back  on  the  knee,  while  tiring. 

□  Flinched  when  the  shot  was  fired. 

□  Bucked  the  rifle. 

□  Jerked  the  trigger. 

Improper  sight  alignment. 

^Impropc'  night  settings, 

□  Position  slipping. 

□  Opened  stance  in  standing  position. 

□  Closed  stance  in  standing  position. 


i  Increased  wind  ve'ooity, 

J  Decreased  wind  velocity, 
i  Wind  is  blowing  from  the  left  but  inadequate  sight 
adjuslnienlB  were  made  to  coinpensale. 

1  Wind  is  blow'i.rig  Irom  the  rigtil  but  inadequate  s'^lit 
adjustments  vrere  made  to  oompensele. 

1  Not  executing  proper  follow-through. 

J  Moved  forward  hand  forward  on  the  handguand,  while  firing. 

1  Moved  forward  hand  backward  on  the  handguard,  v/hite  firing. 
Imprope'  breathing  control. 

Did  not  attain  natural  point  H  aim. 

1  Moved  the  buttstock  placement  tower  on  the  shoulder,  while  firing, 

1  Moved  the  buttstock  placement  higher  on  the  shoulder,  white  firing, 
{  Moved  forwatd  elbov/ torward  on  the  knee,  while  filing, 
j  Moved  torward  elbov/  back  on  the  knee,  while  tiring, 

!  Flinched  v/hen  the  shot  was  fired, 
i  Bucked  the  rifie. 

1  Jerked  tlie  trigger. 

Improper  sight  atignmsnt. 

Improper  sight  settings, 
j  Po.c.iti.on  slipping. 

I  Opened  stance  in  .standing  position. 

1  Closed  stance  In  standing  position. 
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U  Increased  wind  velocity. 

U  Decreased  wind  velocin/. 

U  Wmd  is  blowing  from  the  left  but  inadequate  sight 
adjustinenls  v/ere  made  to  compensate. 

I  □  Wind  is  blowing  Irerri  ttie  right  but  inadequate  sight 
adjustments  v/ere  made  to  compensate. 

□  Not  executing  proper  follow-through, 

□  Moved  forward  hand  forward  on  the  handguard,  while  firing, 

□  Moved  forward  hand  backward  on  the  handguard,  vrhile  firing. 

^  (^Improper  breathing  control, 

^  ^Did  not  attain  natural  point  if  aim. 

□  Moved  the  buttstock  placement  lower  on  the  shoulder,  while  firing. 

□  Moved  the  buttstock  placement  higher  cn  the  shoulder,  v/hite  tiring, 

□  Moved  forward  elbov/ toiward  on  the  knee,  while  Hring, 

□  Moved  forward  elbov/  back  on  the  knee,  white  tiring. 

□  Flinched  v/hen  the  shot  was  fired. 

D  Bucked  the  rifle. 

C  Jerked  the  trigger. 

□  Improper  sight  alignment, 

^  ^Plmprnpcr  sight  setting,'-., 

□  Position  sloping. 

□  Opened  stance  in  standing  position, 

□  Closed  stance  in  standing  position. 


Li  Increased  wind  ve’ccity 
U  Decreased  'wind  velocihy. 

U  Wnd  is  blowing  from  the  left  but  inadequate  sight 
adjuslmenls  v/eie  made  to  ooinperisala, 

□  Wind  is  blowing  Irorii  the  rigiil  but  inadoquala  sight 
adjustments  v/ere  made  to  compen.sete, 

□  Not  executing  proper  follow-through, 

□  Moved  forward  hand  forward  on  the  handguard,  while  lirtng. 

□  Moved  forward  hand  backward  cn  the  handguard,  v/hile  firing. 
(i?lmprope' breathing  control. 

□  Did  not  attain  natural  point  H  aim. 

□  Moved  the  buttstock  placement  lower  on  the  shoulder,  while  firing. 

□  Moved  the  butlstock  placement  higher  on  the  shoulder,  v/hile  firing, 

□  Moved  lorwaid  eloov/  forward  on  the  knee,  while  hring, 

□  Moved  fciwatd  elbov/  bask  on  tlie  knee,  while  tiring. 

□  Flinched  when  the  shot  was  tired, 

□  Buokad  tne  ntle. 

□  Jerked  the  trigger. 

□  Improper  sight  alignment. 

□  Improper  sight  settings, 

□  Position  sloping. 

□  Opened  stance  in  standing  position. 

□  Closed  stance  in  standing  position. 


Knowledge,  Models  and  Tools  to  Improve  the  Effectiveness  of  Naval  Distance  Learning  185 


□  Increased  wind  velocity. 

U  Decreased  wind  velocfry. 

□  Wind  is  blowing  from  the  left  but  inadequate  sight 
adjustmenlK  v«r9  inada  to  compsiisate. 

□  Wind  is  blowing  Irotn  Itie  rigfrt  but  inadequate  siglit 
adjustments  were  made  to  compensate. 

□  Not  executing  proper  foliow-through. 

□  Moved  forward  hand  forward  on  the  handguand,  while  firing. 

□  Mo'/ed  forward  hand  backward  on  the  handguard,  while  firing. 
Improper  breathing  control. 

□  Did  not  attain  natural  point  if  aim. 

□  Moved  the  buttstock  placement  tower  on  the  shoulder,  while  firing, 

□  Moved  the  buttstock  placement  higher  on  the  shoulder,  white  firing. 

□  Moved  forward  elbov/  forward  on  the  knee,  white  firing. 

□  Moved  forward  elbov/  bach  on  fte  knee,  while  firing. 

□  Flinched  v/hen  the  shol  w'as  fired. 

D  Bucked  the  rifle. 

□  Jerked  the  trigger. 

□  Improper  sight  alignment. 

□  Improper  sight  seHing.5. 

□  Position  slipping. 

□  Opened  stance  in  standing  position. 

□  Closed  stance  in  standing  position. 
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APPENDIX  E 

SHOT  GROUP  PATTERN  ANALYSIS  TASK  (PILOT  STUDY  1) 


Knowledge,  Models  and  Tools  to  Improve  the  Effectiveness  of  Naval  Distance  Learning  189 


"Giroupt 


34  i»  r?  *  0  <>  13  IX 

iKcims 


1.  Look  at  the  difference  between  shot  group  1  and  shot  groups.  What  are  the  most  likely  adjustment(s)  the  shooter 
made  between  the  two  groups?  Please  explain  your  answer. 

Adjustment(s): 


Explanation: 


2.  Look  at  shot  group  2  above. 

Does  this  shot  group  show 

a)  proper  marksmanship  fundamentals?  □  Yes 
Why  /  Why  not? 


b)  correct  sight  adjustment? 
Why  /  Why  not? 
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APPENDIX  F 

SHOT  GROUP  DEPICTION  TASK  (PILOT  STUDY  1) 
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INSTRUCTIONS 

Read  each  condition  below,  On  the  target  grid  to  the  left,  plot  five  bullets  that  reflects  the  condition  at  300  yards  in  the 
kneeling  position.  Number  each  shot  to  show  the  shot  sequence. 
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APPENDIX  G 

MAPPING  BETWEEN  SUSTAINMENT-LEVEL  (SLR)  AND  ENTRY-LEVEL 
(ELR)  RIFLE  MARKSMANSHIP  SCORES 


Expert 

Sharpshooter 

Marksman 

Unqualified 

SLR 

ELR 

SLR  ELR 

SLR 

ELR  : 

SLR 

ELR 

65 

250 

39  218 

34 

208 

24 

183 

64 

248 

38  216 

33 

206 

23 

176 

63 

247 

37  214 

32 

204 

22 

169 

62 

246 

36  212 

31 

202 

21 

162 

61 

245 

35  210 

30 

200 

20 

155 

60 

244 

29 

198 

19 

148 

59 

242 

28 

196 

18 

141 

58 

241 

27 

194 

17 

134 

57 

240 

26 

192 

16 

127 

56 

239 

25 

190 

15 

120 

55 

238 

14 

113 

54 

236 

13 

106 

53 

235 

12 

99 

52 

234 

i 

11 

92 

51 

233 

10 

85 

50 

232 

9 

78 

49 

230 

8 

71 

48 

229 

7 

64 

47 

228 

6 

57 

46 

227 

5 

50 

45 

226 

4 

43 

44 

224 

3 

36 

43 

223 

2 

28 

42 

222 

1 

22 

41 

221 

40 

220 
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APPENDIX  H 

AUTOMATED  SCORING  FOR  SHOT  GROUP  ANALYSIS  TASK 

(PILOT  STUDY  2) 
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Data  from  113  students  were  collected  for  five  different  errors  having  to  do  with 
Breathing,  Target  Focus,  Flinching,  Bucking,  and  Sight  Adjustment.  Student  records 
were  scored  dynamically,  on  the  fly,  and  the  results  were  compared  with  human 
grading.  The  final  results  are  listed  below: 


Table  II 


Percent  Agreement  Between  Human  Rater  and 
Automated  Scoring  System  (N  =  113) 


Type 

Percent 

Breathing 

91.2 

Target  Focus 

88.5 

Flinching 

96.5 

Bucking 

94.7 

Sight  Adjustment 

88.5 

In  order  to  score  student  records  on  the  fly,  various  statistical  measures  of  the  shot 
groups  were  taken  into  account  including  the  center  of  mass  of  the  shot  group  and 
its  distance  from  the  center  (0,0)  as  well  as  the  shot  group's  orientation,  maximum 
and  minimum  (highest  and  lowest  shots  and  the  horizontal  and  vertical  ranges),  the 
mean  radius,  area  of  dispersion,  and  finally  the  standard  deviation. 

These  measures  were  then  compared  with  expert  maps  and  scores  were  calculated. 
There  are  two  factors  that  play  a  vital  role  in  dynamic  scoring:  statistical  measures 
and  criteria  applied  in  order  to  compare  student  maps  with  the  experts.  On  the  other 
hand,  in  human  grading,  the  grader  uses  his  or  her  general  knowledge  and 
understanding  of  the  problems  as  well  as  his  or  her  experience  in  the  field. 

One  factor  that  may  contribute  to  the  slightly  different  results  is  the  expert  map  in 
the  case  of  dynamic  scoring,  which  puts  more  restrictions  in  terms  of  the  location 
(coordinates)  of  each  shot  to  be  acceptable.  The  effect  of  the  expert  map  is  bigger  in 
more  complicated  shot  patterns  like  the  Target  Focus  and  Sight  Adjustment 
Problems  (10.6%)  compared  to  more  straightforward  cases  like  Bucking  (5.3%)  and 
Flinching  (3.5%).  In  the  case  of  human  grading,  there  is  no  particular  expert  map  to 
follow.  Another  factor  is  specific  criteria  that  the  algorithm  has  to  follow  in  scoring 
on  the  fly,  and  how  close  this  criterion  is  to  the  vision  of  an  expert  who  does  the 
grading. 


2 


Figure  II.  Shot  group  measures. 

Measures 

Center  of  the  shot  group  (CSG) 


CSG  is  the  center  of  mass  (CM)  of  the  shot  group.  CSG 
the  expert  and  the  student. 
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Constant  Error 

This  is  the  distance  between  the  two  centers  of  mass  (student  and  expert).  It  is  better 
known  as  the  "Constant  Error." 

DCMsg  =  ^|(x^ -x^  +iy^ 

For  the  purpose  of  this  report,  a  shot  group  may  be  considered  functionally  accurate 
if  the  constant  error,  DCMsg  is  less  than  or  equal  to  6. 


Shooting  Error 


The  average  of  the  separate  straight  lines  between  each  student's  shot  and  the  CM  of 
the  expert  shots. 


DCMs  = 


^  expert  )  (Ti  T CM  expert  ) 


n 


DCMs  is  a  useful  measure  of  marksmanship  accuracy  because  it  gives  information 
on  whether  each  shot  actually  comes  close  to  the  center  of  the  target. 

Precision,  The  mean  Radius:  (Fig.  1) 

MR- 

n 

Refers  to  the  average  of  the  distances  between  the  CSG  and  each  shot.  MR  gives  the 
shooter  important  information  about  the  overall  shot  group  tightness.  Shot  group 
tightness  is  a  popular  term  for  marksmanship  precision  and  is  the  magnitude  of 
dispersion  of  a  group  of  shots. 

In  scoring,  we  have  to  compare: 


MRstudent  ^  MRexpert 


the  max  permissible  value  for  |  MRstudent  -  MRexpert  |  ^  6. 

Area  of  Dispersion  (AD)  and  Diagonal  of  Dispersion  (DD)  (see  Figure  II) 
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( ^Max  ^Min  ){y Max 


y  Min  ) 


^  Max  ■  ^  Min  )  ^  (Y  Max  "  Y  Min  ) 

For  each  expert  and  student  these  parameters  are  calculated  and  compared. 

ADstudent  ^  ADexpert 
DDstudent  ^  DDexpert 

Standard  deviation: 

a)  Horizontal  Component,  Sh 


Sh  = 


n  - 1 


b)  Vertical  Component,  Sv 


Sv  = 


c)  Radial  Component,  Sr 

Sr=  ^(s^r+is^Y 

Standard  deviation  is  a  very  efficient  measure  of  variable  error. 

Scoring  Criteria  for  Shot-Group  Task 

Breathing  problem  (does  not  fire  during  the  natural  respiratory  pause). 

The  following  conditions  must  be  satisfied: 

1-  Shots  should  be  vertically  lined  up  around  the  center  line  x  =  0  with  (+1  unit) 
away. 
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2-  ADstudent  VS  ADexpert  should  be  Comparable  (  1 - <  25) 

•‘^^experl 

3-  Highs  AND  lows  of  the  student  and  expert  shots  should  not  be  more  than  1 
unit. 


Bucking  the  rifle: 

1-  All  shots  must  be  located  at  the  third  quarter. 

2-  Shots  must  be  close  together  so  that  ADstudent  vs  ADexpert  should  be 

AD 

comparable  (  1 - <  25) 


Flinching: 

1-  All  shots  must  be  located  at  the  first  quarter. 


Not  focusing  on  front  sight  tip: 

1-  All  shots  must  be  well  distributed  around  the  center  of  the  target  (0,0) 

2- MRstudent  VS  MRexpert  must  be  Comparable. 

3-  Shots  must  be  close  together  so  that  ADstudent  vs  ADexpert  should  be 

AD 

comparable  (  1 - <  25) 

expert 


Sight  adjustment: 

1-  With  the  wind  blowing  from  the  right,  all  the  shots  must  appear  in  the  far  left 

from  the  center,  around  the  line  y  =  0. 

2-  The  shots  must  be  as  close  as  possible  so  the  ADstudent  vs  ADexpert  should  be 

AD 

comparable  (  1 - <  25). 

^^txp  ert 
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Data  File  Format 

Column 

Description 

B 

Xs=Xmax  Xmax  of  the  shooter  shots 

C 

Xe=Xmax  Xmax  of  the  expert  shots 

D 

ratio  of  column  B  /  column  C 

E 

Xs=Xmin  Xmin  of  the  shooter  shots 

F 

Xe=Xmax  Xmin  of  the  expert  shots 

G 

ratio  of  column  E  /  column  F 

H-M  repeats  the  same 
for  the  Y 

N 

RHs  refers  to  the  Horizontal  range  for  the  shooter 

O 

RHe  refers  to  the  Horizontal  range  for  the  expert 

P 

ratio  of  Column  N/Column  O 

Q-S  repeats  the  same 
for  the  vertical  ranges 

T 

ADs  is  the  area  covered  by  the  shooter  shots 

U 

ADe  is  the  area  covered  by  the  expert  shots 

V 

ratio  of  Column  T/ Column  U 

W 

Xbar-s  is  the  X  coordinate  of  the  center  of  mass  for  the  shooter  shots 

X 

Xbar-e  is  the  X  coordinate  of  the  center  of  mass  for  the  expert  shots 

Y 

ratio  of  Column  W/ Column  X 

Z-AB  repeats  the  same 
for  the  Y  coordinates 

AC 

MRs  is  the  mean  Radius  of  the  shooter  shots 

AD 

MRe  is  the  mean  Radius  of  the  expert  shots 

AE 

ratio  of  Column  AC/  Column  AD 

AF 

dsFromCenter  refers  to  the  distance  from  the  center  of  mass  to  point(0,0) 
of  the  shooter  shots 

AG 

deFromCenter  refers  to  the  distance  from  the  center  of  mass  to  point(0,0) 
of  the  expert  shots 

AH 

ratio  of  column  AF  /  column  AG 

AI 

alphas  is  the  angle  between  horizontal  line  (where  y=0  and  x>0)  and  the 
line  from  the  center  of  mass  and  (0,0)  for  the  shooter. 

AJ 

alphaE  is  the  angle  between  horizontal  line  (where  y=0  and  x>0)  and  the 
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line  from  the  center  of  mass  and  (0,0)  for  the  expert. 

AK  ratio  of  Column  Al/Column  AJ 

AL  StandardDev_Horizontal 

AM  StandardDev_Vertical 

AN  StandardDev_RadiaI 

AO  Scores 

AP  Details 

Note. 

1.  s  means  the  shooter  or  the  student,  and  e  means  the  expert. 

2.  With  the  pair  (dsFromCenter,  alphaS)  you  can  find  the  exact  orientation  of  the  center  of  mass  with 
respect  to  where  the  quarter  is  located.  This  information  is  particularly  important  for  scoring  Sight 
Adjustment  and  Flinching. 
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APPENDIX  I 

SHOT  GROUP  TASK  SCORING  RUBRIC 
(PILOT  STUDY  2) 


Shooter  problem 

Score 

Criteria 

Breathing 

1 

If  >  4  shots  are  within  the  middle  two  columns  in  a  vertical  pattern 

0 

If  shot  pattern  does  not  follow  the  "1"  pattern 

Flinching 

1 

If  >  4  shots  fall  in  the  upper  right  quadrant 

0 

If  shot  pattern  does  not  follow  the  "1"  pattern 

Focus  on  Target 

1 

If  shots  form  a  circular  pattern  around  the  target 

0 

If  shot  pattern  does  not  follow  the  "1"  pattern 

Bucking 

1 

If  >  4  shots  fall  in  the  lower  left  quadrant 

0 

If  shot  pattern  does  not  follow  the  "1"  pattern 

Sight  Adjustment 

1 

If  >  4  shots  fall  clustered  to  the  left  of  the  target,  within  the  two 
middle  rows 

0 

If  shot  pattern  does  not  follow  the  "1"  pattern 
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APPENDIX  J 

CRITERION  KNOWLEDGE  MAPS  FOR  MARKSMANSHIP  (SCREEN  SHOTS) 
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Expert  map  for  scoring  in  MarcVMay  2003  Expert  2 
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Expert  map  for  scoring  in  MarcVlVIay  2003  Expert  4 


Expert  map  for  scoring  in  March/May  2003  Expert  5 
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APPENDIX  K 

CRITERION  KNOWLEDGE  MAPS  FOR  MARKSMANSHIP  (PROPOSITIONS) 
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Expert 

Proposi¬ 
tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

Expert 

Nov./Dec. 

2002 

1 

7  factors  common  to  all 
shooting  positions 

helps 

stable  firing  position 

2 

7  factors  common  to  all 
shooting  positions 

requires 

breath  control 

3 

7  factors  common  to  all 
shooting  positions 

requires 

stock  weld  placement 

4 

7  factors  common  to  all 
shooting  positions 

requires 

elbow  placement 

5 

7  factors  common  to  all 
shooting  positions 

requires 

forward  hand  placement 

6 

7  factors  common  to  all 
shooting  positions 

requires 

grip  of  firing  hand 

7 

7  factors  common  to  all 
shooting  positions 

requires 

muscular  relaxation 

8 

7  factors  common  to  all 
shooting  positions 

requires 

placement  of  buttstock  in 
shoulder 

9 

aiming  process 

leads  to 

fundamentals  of 
marksmanship 

10 

breath  control 

improves 

sight  picture 

11 

breath  control 

leads  to 

fundamentals  of 
marksmanship 

12 

breath  control 

part  of 

fundamentals  of 
marksmanship 

13 

stock  weld  placement 

affects 

sight  alignment 

14 

stock  weld  placement 

leads  to 

stable  firing  position 

15 

elbow  placement 

leads  to 

stable  firing  position 

16 

eye  on  front  sight  post 

part  of 

sight  picture 

17 

eye  relief 

affects 

sight  alignment 

18 

eye  relief 

affects 

sight  picture 

19 

finger  placement 

affects 

trigger  squeeze 

20 

follow-through 

part  of 

fundamentals  of 
marksmanship 

21 

forward  hand  placement 

leads  to 

stable  firing  position 

22 

grip  of  firing  hand 

affects 

finger  placement 

23 

grip  of  firing  hand 

affects 

trigger  squeeze 

24 

muscular  relaxation 

increases 

stable  firing  position 
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Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

25 

muscular  tension 

increases 

stable  firing  position 

26 

natural  respiratory  pause 

part  of 

breath  control 

27 

placement  of  buttstock  in 
shoulder 

helps 

stable  firing  position 

28 

sight  alignment 

part  of 

aiming  process 

29 

sight  alignment 

part  of 

fundamentals  of 
marksmanship 

30 

sight  picture 

part  of 

fundamentals  of 
marksmanship 

31 

sight  picture 

part  of 

aiming  process 

32 

stable  firing  position 

increases 

muscular  relaxation 

33 

stable  firing  position 

leads  to 

fundamentals  of 
marksmanship 

34 

stable  firing  position 

requires 

muscular  relaxation 

35 

trigger  control 

leads  to 

fundamentals  of 
marksmanship 

36 

trigger  squeeze 

affects 

trigger  control 

37 

trigger  squeeze 

during 

natural  respiratory  pause 

38 

bone  support 

leads  to 

stable  firing  position 

39 

stable  firing  position 

requires 

bone  support 

40 

breath  control 

increases 

accuracy 

41 

elbow  placement 

increases 

accuracy 

42 

forward  hand  placement 

increases 

accuracy 

43 

grip  of  firing  hand 

increases 

accuracy 

44 

muscular  relaxation 

increases 

accuracy 

45 

placement  of  buttstock  in 
shoulder 

increases 

accuracy 

1 

natural  respiratory  pause 

leads  to 

consistency 

2 

natural  respiratory  pause 

part  of 

breath  control 

3 

breath  control 

part  of 

fundamentals  of 
marksmanship 

4 

follow-through 

leads  to 

consistency 

5 

follow-through 

helps 

consistency 
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Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

6 

follow-through 

part  of 

fundamentals  of 
marksmanship 

7 

breath  control 

part  of 

7  factors  common  to  all 
shooting  positions 

8 

consistency 

helps 

aiming  process 

9 

aiming  process 

part  of 

fundamentals  of 
marksmanship 

10 

sight  alignment 

part  of 

aiming  process 

11 

eye  relief 

affects 

sight  alignment 

12 

eye  relief 

part  of 

aiming  process 

13 

eye  relief 

affects 

sight  picture 

14 

eye  on  front  sight  post 

part  of 

aiming  process 

15 

eye  on  front  sight  post 

affects 

sight  alignment 

16 

eye  on  front  sight  post 

affects 

sight  picture 

17 

natural  point  of  aim 

part  of 

aiming  process 

18 

eye  relief 

part  of 

7  factors  common  to  all 
shooting  positions 

19 

stock  weld  placement 

affects 

sight  alignment 

20 

stock  weld  placement 

affects 

eye  relief 

21 

stock  weld  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

22 

placement  of  buttstock  in 
shoulder 

affects 

stock  weld  placement 

23 

placement  of  buttstock  in 
shoulder 

affects 

rear  elbow  placement 

24 

placement  of  buttstock  in 
shoulder 

affects 

forward  elbow  placement 

25 

placement  of  buttstock  in 
shoulder 

part  of 

7  factors  common  to  all 
shooting  positions 

26 

rear  elbow  placement 

leads  to 

stable  firing  position 

27 

forward  elbow  placement 

part  of 

stable  firing  position 

28 

rear  elbow  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

29 

forward  elbow  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

30 

rear  elbow  placement 

affects 

grip  of  firing  hand 
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Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

31 

grip  of  firing  hand 

part  of 

stable  firing  position 

32 

natural  point  of  aim 

affects 

stable  firing  position 

33 

controlled  muscular 
tension 

part  of 

stable  firing  position 

34 

bone  support 

part  of 

stable  firing  position 

35 

bone  support 

helps 

controlled  muscular  tension 

36 

muscular  relaxation 

part  of 

3  elements  of  a  good  shooting 
position 

37 

bone  support 

part  of 

3  elements  of  a  good  shooting 
position 

38 

natural  point  of  aim 

part  of 

3  elements  of  a  good  shooting 
position 

39 

grip  of  firing  hand 

affects 

finger  placement 

40 

sight  picture 

part  of 

aiming  process 

41 

muscular  relaxation 

affects 

grip  of  firing  hand 

42 

controlled  muscular 
tension 

affects 

grip  of  firing  hand 

1 

grip  of  firing  hand 

affects 

trigger  control 

2 

grip  of  firing  hand 

leads  to 

trigger  control 

3 

controlled  muscular 
tension 

helps 

grip  of  firing  hand 

4 

controlled  muscular 
tension 

decreases 

bone  support 

5 

bone  support 

helps 

stable  firing  position 

6 

stock  weld  placement 

leads  to 

stable  firing  position 

7 

placement  of  buttstock  in 
shoulder 

affects 

stock  weld  placement 

8 

stable  firing  position 

helps 

follow-through 

9 

stable  firing  position 

part  of 

fundamentals  of 
marksmanship 

10 

consistency 

part  of 

fundamentals  of 
marksmanship 

11 

aiming  process 

part  of 

fundamentals  of 
marksmanship 

12 

natural  respiratory  pause 

affects 

aiming  process 
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• 

Expert 

Proposi¬ 
tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

13 

trigger  control 

affects 

aiming  process 

14 

natural  respiratory  pause 

helps 

natural  point  of  aim 

15 

stock  weld  placement 

part  of 

eye  relief 

• 

16 

natural  point  of  aim 

affects 

aiming  process 

Expert  3 

1 

eye  relief 

affects 

eye  on  front  sight  post 

2 

stock  weld  placement 

affects 

eye  relief 

• 

3 

stock  weld  placement 

affects 

eye  on  front  sight  post 

4 

eye  relief 

affects 

sight  picture 

5 

aiming  process 

requires 

eye  on  front  sight  post 

6 

aiming  process 

leads  to 

consistency 

• 

7 

aiming  process 

requires 

sight  picture 

8 

aiming  process 

requires 

sight  alignment 

9 

eye  on  front  sight  post 

follows 

sight  picture 

10 

forward  hand  placement 

affects 

sight  picture 

• 

11 

forward  hand  placement 

affects 

sight  alignment 

12 

placement  of  buttstock  in 
shoulder 

affects 

sight  alignment 

13 

sight  picture 

follows 

sight  alignment 

• 

14 

placement  of  buttstock  in 
shoulder 

affects 

sight  picture 

15 

forward  hand  placement 

leads  to 

consistency 

16 

placement  of  buttstock  in 
shoulder 

leads  to 

consistency 

• 

17 

stock  weld  placement 

requires 

controlled  muscular  tension 

18 

grip  of  firing  hand 

requires 

controlled  muscular  tension 

19 

grip  of  firing  hand 

affects 

trigger  control 

20 

grip  of  firing  hand 

affects 

finger  placement 

• 

21 

finger  placement 

affects 

trigger  control 

22 

trigger  control 

requires 

controlled  muscular  tension 

23 

trigger  control 

leads  to 

consistency 

A 

24 

natural  respiratory  pause 

leads  to 

consistency 

V 

25 

muscular  relaxation 

leads  to 

consistency 

26 

muscular  relaxation 

part  of 

stable  firing  position 

228 


CSE  Deliverable 


Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

27 

muscular  relaxation 

helps 

stable  firing  position 

28 

muscular  relaxation 

helps 

follow-through 

29 

follow-through 

helps 

consistency 

30 

stable  firing  position 

leads  to 

consistency 

31 

7  factors  common  to  all 
shooting  positions 

leads  to 

stable  firing  position 

32 

bone  support 

part  of 

stable  firing  position 

33 

bone  support 

increases 

stable  firing  position 

34 

bone  support 

helps 

follow-through 

35 

stock  weld  placement 

leads  to 

consistency 

36 

stock  weld  placement 

helps 

consistency 

1 

aiming  process 

requires 

eye  relief 

2 

aiming  process 

part  of 

fundamentals  of 
marksmanship 

3 

natural  point  of  aim 

affects 

aiming  process 

4 

sight  picture 

part  of 

aiming  process 

5 

sight  alignment 

part  of 

aiming  process 

6 

eye  on  front  sight  post 

part  of 

aiming  process 

7 

breath  control 

leads  to 

natural  respiratory  pause 

8 

breath  control 

part  of 

fundamentals  of 
marksmanship 

9 

trigger  control 

part  of 

fundamentals  of 
marksmanship 

10 

7  factors  common  to  all 
shooting  positions 

requires 

fundamentals  of 
marksmanship 

11 

7  factors  common  to  all 
shooting  positions 

requires 

bone  support 

12 

7  factors  common  to  all 
shooting  positions 

requires 

stable  firing  position 

13 

muscular  relaxation 

increases 

bone  support 

14 

forward  elbow  placement 

part  of 

bone  support 

15 

rear  elbow  placement 

part  of 

bone  support 

16 

stable  firing  position 

uses 

forward  hand  placement 
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Expert  Proposi-  Source  Concept  Relationship  Destination  Concept 

tion  No. 


17 

forward  hand  placement 

helps 

placement  of  buttstock  in 
shoulder 

18 

placement  of  buttstock  in 
shoulder 

affects 

stock  weld  placement 

1 

7  factors  common  to  all 
shooting  positions 

affects 

forward  elbow  placement 

2 

forward  elbow  placement 

affects 

eye  relief 

3 

forward  elbow  placement 

leads  to 

bone  support 

4 

bone  support 

affects 

eye  relief 

5 

bone  support 

helps 

eye  relief 

6 

eye  relief 

affects 

fundamentals  of 
marksmanship 

7 

fundamentals  of 
marksmanship 

affects 

muscular  relaxation 

8 

muscular  relaxation 

affects 

placement  of  buttstock  in 
shoulder 

9 

eye  relief 

affects 

finger  placement 

10 

finger  placement 

affects 

follow-through 

11 

follow-through 

affects 

fundamentals  of 
marksmanship 

12 

fundamentals  of 
marksmanship 

affects 

grip  of  firing  hand 

13 

placement  of  buttstock  in 
shoulder 

affects 

rear  elbow  placement 

14 

rear  elbow  placement 

affects 

stable  firing  position 

15 

stable  firing  position 

affects 

stock  weld  placement 

16 

stock  weld  placement 

affects 

consistency 

17 

grip  of  firing  hand 

affects 

muscular  relaxation 

18 

muscular  relaxation 

affects 

controlled  muscular  tension 

19 

controlled  muscular 
tension 

affects 

placement  of  buttstock  in 
shoulder 

20 

consistency 

affects 

natural  point  of  aim 

21 

forward  elbow  placement 

during 

follow-through 

22 

follow-through 

affects 

natural  point  of  aim 
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Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

23 

forward  elbow  placement 

helps 

7  factors  common  to  all 
shooting  positions 

24 

7  factors  common  to  all 
shooting  positions 

helps 

aiming  process 

25 

7  factors  common  to  all 
shooting  positions 

helps 

bone  support 

26 

aiming  process 

affects 

natural  point  of  aim 

28 

eye  relief 

helps 

sight  alignment 

29 

sight  alignment 

helps 

sight  picture 

30 

sight  picture 

helps 

trigger  control 

31 

trigger  control 

helps 

consistency 

1 

eye  on  front  sight  post 

part  of 

aiming  process 

2 

sight  alignment 

part  of 

aiming  process 

3 

sight  picture 

part  of 

aiming  process 

4 

eye  relief 

affects 

sight  picture 

5 

eye  relief 

affects 

sight  alignment 

6 

stock  weld  placement 

affects 

eye  relief 

7 

natural  point  of  aim 

part  of 

aiming  process 

8 

breath  control 

part  of 

aiming  process 

9 

natural  respiratory  pause 

part  of 

aiming  process 

10 

bone  support 

part  of 

3  elements  of  a  good  shooting 
position 

11 

breath  control 

part  of 

3  elements  of  a  good  shooting 
position 

12 

controlled  muscular 
tension 

part  of 

3  elements  of  a  good  shooting 
position 

13 

controlled  muscular 
tension 

part  of 

muscular  relaxation 

14 

muscular  relaxation 

part  of 

controlled  muscular  tension 

15 

placement  of  buttstock  in 
shoulder 

part  of 

7  factors  common  to  all 
shooting  positions 

16 

forward  hand  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

17 

forward  elbow  placement 

part  of 

7  factors  common  to  all 
shooting  positions 
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Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

18 

forward  hand  placement 

affects 

forward  elbow  placement 

19 

forward  elbow  placement 

affects 

placement  of  buttstock  in 
shoulder 

20 

placement  of  buttstock  in 
shoulder 

affects 

rear  elbow  placement 

21 

rear  elbow  placement 

helps 

grip  of  firing  hand 

22 

grip  of  firing  hand 

affects 

finger  placement 

23 

finger  placement 

during 

trigger  control 

24 

trigger  control 

requires 

finger  placement 

25 

finger  placement 

helps 

trigger  control 

26 

trigger  control 

helps 

consistency 

27 

7  factors  common  to  all 
shooting  positions 

leads  to 

consistency 

28 

consistency 

requires 

7  factors  common  to  all 
shooting  positions 

29 

3  elements  of  a  good 
shooting  position 

leads  to 

consistency 

30 

aiming  process 

increases 

consistency 

1 

finger  placement 

requires 

consistency 

2 

follow-through 

leads  to 

consistency 

3 

finger  placement 

leads  to 

trigger  control 

4 

follow-through 

helps 

trigger  control 

5 

grip  of  firing  hand 

requires 

consistency 

6 

grip  of  firing  hand 

leads  to 

finger  placement 

7 

finger  placement 

uses 

controlled  muscular  tension 

8 

grip  of  firing  hand 

requires 

controlled  muscular  tension 

9 

controlled  muscular 
tension 

helps 

trigger  control 

10 

trigger  control 

requires 

grip  of  firing  hand 

11 

trigger  control 

requires 

consistency 

12 

natural  point  of  aim 

uses 

muscular  relaxation 

13 

breath  control 

leads  to 

stable  firing  position 

14 

muscular  relaxation 

leads  to 

stable  firing  position 

15 

muscular  relaxation 

helps 

breath  control 
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Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

16 

natural  point  of  aim 

uses 

bone  support 

17 

bone  support 

leads  to 

stable  firing  position 

18 

stable  firing  position 

helps 

natural  point  of  aim 

19 

breath  control 

helps 

aiming  process 

20 

breath  control 

uses 

natural  respiratory  pause 

21 

natural  respiratory  pause 

helps 

aiming  process 

22 

controlled  muscular 
tension 

helps 

aiming  process 

23 

fundamentals  of 
marksmanship 

requires 

trigger  control 

24 

stock  weld  placement 

requires 

controlled  muscular  tension 

25 

fundamentals  of 
marksmanship 

requires 

breath  control 

26 

sight  picture 

affects 

eye  on  front  sight  post 

27 

eye  on  front  sight  post 

part  of 

aiming  process 

28 

sight  alignment 

affects 

eye  on  front  sight  post 

29 

sight  picture 

part  of 

aiming  process 

30 

fundamentals  of 
marksmanship 

requires 

aiming  process 

31 

fundamentals  of 
marksmanship 

requires 

sight  alignment 

32 

fundamentals  of 
marksmanship 

requires 

sight  picture 

33 

stable  firing  position 

helps 

fundamentals  of 
marksmanship 

34 

stock  weld  placement 

affects 

eye  relief 

35 

stock  weld  placement 

requires 

consistency 

36 

eye  relief 

requires 

consistency 

37 

rear  elbow  placement 

requires 

consistency 

38 

placement  of  buttstock  in 
shoulder 

requires 

consistency 

39 

forward  hand  placement 

requires 

consistency 

40 

stock  weld  placement 

leads  to 

stable  firing  position 

41 

forward  elbow  placement 

affects 

stable  firing  position 

42 

sight  alignment 

part  of 

aiming  process 
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• 

Expert 

Proposi¬ 
tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

43 

eye  relief 

affects 

aiming  process 

44 

7  factors  common  to  all 
shooting  positions 

helps 

stable  firing  position 

45 

7  factors  common  to  all 
shooting  positions 

affects 

fundamentals  of 
marksmanship 

46 

forward  hand  placement 

affects 

forward  elbow  placement 

47 

forward  hand  placement 

leads  to 

stable  firing  position 

48 

placement  of  buttstock  in 
shoulder 

leads  to 

stable  firing  position 

49 

grip  of  firing  hand 

leads  to 

stable  firing  position 

50 

rear  elbow  placement 

leads  to 

stable  firing  position 

51 

forward  hand  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

52 

placement  of  buttstock  in 
shoulder 

part  of 

7  factors  common  to  all 
shooting  positions 

53 

grip  of  firing  hand 

part  of 

7  factors  common  to  all 
shooting  positions 

54 

rear  elbow  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

55 

stock  weld  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

56 

breath  control 

part  of 

7  factors  common  to  all 
shooting  positions 

57 

natural  point  of  aim 

affects 

aiming  process 

58 

aiming  process 

affects 

natural  point  of  aim 

59 

muscular  relaxation 

part  of 

controlled  muscular  tension 

60 

controlled  muscular 
tension 

part  of 

muscular  relaxation 

61 

muscular  relaxation 

helps 

bone  support 

62 

bone  support 

helps 

muscular  relaxation 

Expert  8 

1 

bone  support 

increases 

stable  firing  position 

2 

controlled  muscular 
tension 

increases 

stable  firing  position 

• 

3 

muscular  relaxation 

increases 

stable  firing  position 

4 

natural  point  of  aim 

part  of 

3  elements  of  a  good  shooting 
position 

234 


CSE  Deliverable 


Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

5 

bone  support 

part  of 

3  elements  of  a  good  shooting 
position 

6 

controlled  muscular 
tension 

part  of 

3  elements  of  a  good  shooting 
position 

7 

muscular  relaxation 

part  of 

3  elements  of  a  good  shooting 
position 

8 

3  elements  of  a  good 
shooting  position 

helps 

consistency 

9 

7  factors  common  to  all 
shooting  positions 

helps 

consistency 

10 

stock  weld  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

11 

eye  relief 

part  of 

7  factors  common  to  all 
shooting  positions 

12 

placement  of  buttstock  in 
shoulder 

part  of 

7  factors  common  to  all 
shooting  positions 

13 

breath  control 

part  of 

7  factors  common  to  all 
shooting  positions 

14 

rear  elbow  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

15 

grip  of  firing  hand 

part  of 

7  factors  common  to  all 
shooting  positions 

16 

forward  hand  placement 

part  of 

7  factors  common  to  all 
shooting  positions 

17 

placement  of  buttstock  in 
shoulder 

affects 

eye  relief 

18 

eye  relief 

affects 

placement  of  buttstock  in 
shoulder 

19 

breath  control 

part  of 

natural  respiratory  pause 

20 

natural  respiratory  pause 

helps 

breath  control 

21 

breath  control 

part  of 

fundamentals  of 
marksmanship 

22 

trigger  control 

part  of 

fundamentals  of 
marksmanship 

23 

aiming  process 

part  of 

fundamentals  of 
marksmanship 

24 

fundamentals  of 
marksmanship 

helps 

consistency 
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Expert  Proposi¬ 

tion  No. 

Source  Concept 

Relationship 

Destination  Concept 

25 

stock  weld  placement 

affects 

aiming  process 

26 

eye  relief 

affects 

aiming  process 

27 

eye  on  front  sight  post 

affects 

consistency 

28 

natural  respiratory  pause 

affects 

consistency 

29 

eye  on  front  sight  post 

part  of 

sight  alignment 

30 

sight  alignment 

part  of 

aiming  process 

31 

sight  picture 

part  of 

aiming  process 

32 

controlled  muscular 
tension 

part  of 

grip  of  firing  hand 

33 

muscular  relaxation 

part  of 

grip  of  firing  hand 

34 

placement  of  buttstock  in 
shoulder 

helps 

stable  firing  position 

35 

breath  control 

helps 

stable  firing  position 

36 

rear  elbow  placement 

helps 

stable  firing  position 

37 

grip  of  firing  hand 

helps 

stable  firing  position 

38 

forward  hand  placement 

helps 

stable  firing  position 

39 

finger  placement 

leads  to 

grip  of  firing  hand 
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APPENDIX  L 

BASIC  MARKSMANSHIP  KNOWLEDGE  MEASURE 

(STUDY  2) 
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Rifle  Marksmanship  Study 
Prior  Knowledge 
March  2003 


Matching 

Directions:  On  the  line  to  the  left  of  each  definition,  write  the  letter  of  the  term  which 
best  matches  the  definition.  Each  term  may  be  used  only  once.  There  are  more  terms 
than  definitions. 


Definition 


Term 


,1.  The  firm  consistent  contact  of  the  cheek  with 
the  weapon's  butt  stock 

2.  The  distance  between  the  rear  sight  aperture 
and  the  aiming  eye 

.3.  The  body's  skeletal  structure  supporting  the 
rifle's  weight 

,4.  The  point  in  the  breathing  cycle  during  which 
the  body  is  most  relaxed,  allowing  the  sights 
to  settle  at  the  natural  point  of  aim 

.5.  The  skillful  manipulation  of  the  trigger  that 
causes  the  rifle  to  fire  without  disturbing  sight 
alignment  or  sight  picture 

6.  "V"  formed  between  the  thumb  and  index 
finger  on  the  trigger 

7.  Continued  application  of  the  fundamentals 
until  the  round  has  exited  the  barrel 


a)  Bone  support 

b)  Eye  relief 

c)  Firing  hand 
placement 

d)  Firm  pistol  grip 

e)  Follow-through 

f)  Muscular  relaxation 

g)  Natural  respiratory 
pause 

h)  Recovery 

i)  Sight  adjustment 

j)  Sight  alignment 

k)  Stable  firing  position 

l)  Stock  weld 

m)  Trigger  control 


,8.  The  process  used  to  adjust  the  rifle  sights  that 
causes  the  rifle  to  shoot  at  the  point  of  aim  at  a 
desired  range 


9.  The  state  of  tension  required  to  properly 
control  the  rifle 


240 


CSE  Deliverable 


Below  are  some  HYPOTHETICAL 
situations.  Indicate  how  much  you  agree 
with  each  statement. 

Strongly 

agree 

Agree 

Strongly 
Disagree  disagree 

41 

If  a  shooter's  muscles  never  got  fatigued  . . . 

10.  ...  it  would  be  OK  to  "muscle"  the  rifle 
into  position. 

□ 

□ 

□ 

□ 

• 

11.  ...  there  would  be  little  need  for  bone 
support. 

Consistency  is  important  in  shooting  because 

□ 

□ 

□ 

□ 

• 

12.  ...  even  very  small  changes  in  body 

position  from  one  shot  to  the  next  can 
result  in  a  poor  shot. 

□ 

□ 

□ 

□ 

13.  ...  it  helps  you  figure  out  what  to  adjust 
shot  to  shot. 

A  very  long  natural  respiratory  pause  (over  15 
seconds)  ... 

□ 

□ 

□ 

□ 

• 

14.  ...  will  improve  the  stability  of  a  shooter's 
position. 

□ 

□ 

□ 

□ 

• 

15.  ...  will  probably  not  affect  the  shooter  too 
much. 

□ 

□ 

□ 

□ 

16.  Given  a  randomly  shaped  target,  which  figure  best  shows  the  correct  aiming 
point? 

a)  Figure  1 

b)  Figure  2 

c)  Figure  3 
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Multiple  Choice 

Directions:  Use  this  picture  when  answering  questions  17  and  18.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


17.  Which  best  describes  the  sight  picture/  sight  alignment  shown  in  the  picture 
above? 

a)  Proper  sight  picture,  proper  sight  alignment 

b)  Proper  sight  picture,  improper  sight  alignment 

c)  Improper  sight  picture,  proper  sight  alignment 

d)  Improper  sight  picture,  improper  sight  alignment 


18.  Given  the  above  sight  picture/ sight  alignment,  which  figure  below  best 
shows  where  the  round  would  strike?  Assume  a  properly  zeroed  rifle,  no 
wind  or  weather  effects,  and  a  properly  fired  weapon  at  300  yards. 

a)  Figure  1 

b)  Figure  2 

c)  Figure  3 

d)  Figure  4 


Figure  1 


Figure  2 


Figure  3 


Figure  4 
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Directions:  Use  this  picture  when  answering  questions  19  and  20.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


19.  Which  best  describes  the  sight  picture/ sight  alignment  shown  in  the  picture 
above? 

a)  Proper  sight  picture,  proper  sight  alignment 

b)  Proper  sight  picture,  improper  sight  alignment 

c)  Improper  sight  picture,  proper  sight  alignment 

d)  Improper  sight  picture,  improper  sight  alignment 


20.  Given  the  above  sight  picture/ sight  alignment,  which  figure  below  best 
shows  where  the  round  would  strike?  Assume  a  properly  zeroed  rifle,  no 
wind  or  weather  effects,  and  a  properly  fired  weapon  at  300  yards. 

a)  Figure  1 

b)  Figure  2 

c)  Figure  3 

d)  Figure  4 


Figure  1 


Figure  2 


Figure  3 


Figure  4 


Knowledge,  Models  and  Tools  to  Improve  the  Effectiveness  of  Naval  Distance  Learning  243 


Directions:  Use  this  picture  when  answering  questions  21  and  22.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


21.  Which  best  describes  the  sight  picture/  sight  alignment  shown  in  the  picture 
above? 

a)  Proper  sight  picture,  proper  sight  alignment 

b)  Proper  sight  picture,  improper  sight  alignment 

c)  Improper  sight  picture,  proper  sight  alignment 

d)  Improper  sight  picture,  improper  sight  alignment 


22.  Given  the  above  sight  picture/ sight  alignment,  which  figure  below  best 
shows  where  the  round  would  strike?  Assume  a  properly  zeroed  rifle,  no 
wind  or  weather  effects,  and  a  properly  fired  weapon  at  300  yards. 

a)  Figure  1 

b)  Figure  2 

c)  Figure  3 

d)  Figure  4 


Figure  1 


Figure  2 


Figure  3 


Figure  4 
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Directions:  Use  these  pictures  when  answering  questions  23  to  25.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


Sight  picture  #1 


Sight  picture  #2 


Sight  picture  #3 


Note:  It  may  be  helpful  to  read  all  3  questions  before  answering. 


23.  Which  sight  picture  above  best  reflects  the 
Marine's  eye  relief  shown  in  the  picture  to  the 
left? 

a)  Sight  picture  #1 

b)  Sight  picture  #2 

c)  Sight  picture  #3 


24.  Which  sight  picture  above  best  reflects  the 
Marine's  eye  relief  shown  in  the  picture  to  the 
left? 

a)  Sight  picture  #1 

b)  Sight  picture  #2 

c)  Sight  picture  #3 


25.  Which  sight  picture  above  best  reflects  the 
Marine's  eye  relief  shown  in  the  picture  to  the 
left? 

a)  Sight  picture  #1 

b)  Sight  picture  #2 

c)  Sight  picture  #3 
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Multiple  Choice 

26.  In  the  sitting  position,  which  direction  does  the  rifle  muzzle  move  when  the  rifle 
is  fired  while  inhaling?  Assume  breathing  from  the  chest. 

a)  Up 

b)  Down 

c)  Left 

d)  Right 

27.  Which  of  the  following  safety  rules  best  enforces  muzzle  awareness? 

a)  Treat  every  weapon  as  if  it  were  loaded. 

b)  Never  point  a  weapon  at  anything  you  do  not  intend  to  shoot. 

c)  Keep  your  finger  straight  and  off  the  trigger  until  you  are  ready  to  fire. 

d)  Keep  the  weapon  on  safe  until  you  intend  to  fire. 

28.  Which  of  the  following  is  the  proper  rear  sight  elevation  setting  when  firing  at 
the  300-yard  line?  Assume  a  zeroed  rifle. 

a)  5 

b)  5/3 

c)  8/3 

d)  8/3-2 

29.  How  far  will  2  clicks  of  the  rear  sight  windage  move  the  strike  of  the  round  at 
500  yards? 

a)  1  inch 

b)  2 1/2  inches 

c)  3  3/4  inches 

d)  5  inches 

30.  Suppose  you  have  perfect  sight  alignment  and  sight  picture  (D-target).  At  the 
moment  of  the  shot,  the  muzzle  moves  UP  1/16  inches  (a  width  of  a  quarter). 
About  how  far  off  the  center  line  do  you  think  the  round  will  strike  at  200  yards? 

a)  A  couple  of  inches  (a  hit,  still  in  the  center  ring) 

b)  Almost  6  inches  (a  hit,  still  in  center  ring,  barely) 

c)  Around  10  inches  (a  miss,  above  black  silhouette  by  almost  an  inch) 

d)  Around  16  inches  (a  miss,  well  above  black) 

31.  Which  reason  best  explains  the  function  of  bone  support? 

a)  To  provide  a  strong  frame  to  absorb  the  rifle  recoil 

b)  To  help  support  the  weight  of  the  rifle 

c)  To  provide  rigid  contact  with  the  ground 

d)  To  allow  the  shooter  to  resist  strong  crosswinds  in  the  field 
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Questions  32-33: 

Triangulate  the  shot  group.  Write  the  w^indage  and  elevation  adjustments  needed  to 
place  the  center  of  the  shot  group  into  the  center  of  the  target.  Specify  the  number  of 
clicks  and  the  direction.  Keep  in  mind  the  rifle  is  set  at  the  initial  sight  setting  (flush 
and  center)  for  each  question. 

UNITS  ARE  IN  CLICKS.  YOU  DO  NOT  HAVE  TO  PERFORM  ANY 
CONVERSIONS. 


PLOT 


2A  IB  e  la  18  24 


32.  FSP:  _ clicks,  □  up  □  down 

RSW:  _ clicks,  □  left  □  right 

UNITS  ARE  IN  CLICKS.  YOU  DO  NOT  HAVE  TO 
PERFORM  ANY  CONVERSIONS. 


?4  1fl  1?  t>  6  1?  ia  ?4 

CLICKS 


33.  FSP:  _ clicks,  □  up  □  down 

RSW:  _ clicks,  □  left  □  right 

UNITS  ARE  IN  CLICKS.  YOU  DO  NOT  HAVE  TO 
PERFORM  ANY  CONVERSIONS. 
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Up  /  Left  /  No 

down  right  Erratic  change 

In  which  direction  will  the  natural  point  of  aim  shift 
(right-handed  shooter,  KNEELING)  . . . 


34. 

...  if  the  left  hand  is  moved  along  the 
handguard? 

□ 

□ 

□ 

□ 

35. 

. . .  if  the  rifle  butt  is  moved  up  or  down  in  the 
shoulder? 

□ 

□ 

□ 

□ 

36. 

...  if  the  left  elbow  is  moved  along  the  knee? 

□ 

□ 

□ 

□ 

37. 

...  if  the  entire  body  is  moved  in  relation  to  the 
target? 

□ 

□ 

□ 

□ 

In  which  direction  will  the  rifle  muzzle  shift  from  shot 
to  shot . . . 

38. 

...  if  the  trigger  finger  is  in  contact  with  the 
receiver  or  pistol  grip  while  firing? 

□ 

□ 

□ 

□ 

39. 

...  if  the  shooter  is  holding  his  breath  too  long? 

□ 

□ 

□ 

□ 

40. 

...  if  the  shooter  has  high  muscle  fatigue? 

□ 

□ 

□ 

□ 

41. 

...  if  the  shooter's  position  is  constantly 

□ 

□ 

□ 

□ 

changing? 
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APPENDIX  M 

SHOT  GROUP  DEPICTION  MEASURE 
(STUDY  2) 
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Rifle  Marksmanship  Study 

Shot  Group  Analysis 

^  March  2003 

Directions:  Each  question  describes  a  particular  shooter  error.  On  the  target  to  the 
right,  plot  five  bullet  strikes  that  reflect  the  conditions  at  300  yards  of  a  right- 
handed  shooter  in  the  kneeling  position.  Unless  otherwise  stated,  assume  no 
®  wind/ weather  effects. 


1. 


Breathing  while  firing 


n 

B 


2. 


Bucking  the  shoulder  into  the 
rifle  stock  while  firing 
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3.  Flinching  the  body  while  firing 
(sudden  small  backward 
movement  of  body) 


4.  Aiming  eye  focused  on  the 
target  instead  of  the  front  sight 
post  while  firing 


< . i  t  ♦  . . . ^ 

2-1  IK  I'  ^  i.  ty  m 

iNCHhy 


5.  Sight  adjustment  problem 
(inadequate  compensation) 
with  wind  blowing  from  the 
right 
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APPENDIX  N 

TRAIT  WORRY  MEASURE 


How  often  do  you  feel  or  think  the  following? 

Almost 

Some- 

Almost 

always 

Often 

times 

never 

Thinking  about  my  score  interferes  with  my  shooting  during 
qualification. 

□ 

□ 

□ 

□ 

|l  freeze  up  during  qualification.  ^  ^ 

□ 

■  I'Q 

During  qualification  1  find  myself  thinking  about  whether  I’ll  ever 
get  through  it. 

□ 

□ 

a 

□ 

[jhe  more  1  think  during  qualification,  the  more  confused  1 

u 

a 

1  seem  to  defeat  myself  during  qualification. 

□ 

□ 

□ 

a 

buring  qualification  1  find  myself  thinking  about  the  - 

Ipnsequences  of  failing.  7  ■ 

□ 

: 

■ '  yf-.: 

□ 

a 

□ 

Thoughts  of  doing  poorly  interfere  with  my  concentration  during 
qualification. 

a 

a 

□ 

a 

buring  qualifiqation  1  get  so  nervous  that  1  forget  skills  and  ,  .  Cv 
Information  1  really  know.  yi  ' 

.  □ 

□ 

□ 
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APPENDIX  O 

STATE  ANXIETY  AND  STATE  WORRY  MEASURE 


Directions:  A  number  of  statements  which  people  have  used  to  describe  themselves 
are  given  below.  Read  each  statement  and  find  the  word  or  phrase  that  best 
indicates  how  you  think  or  feel  right  now.  Circle  the  number  for  your  answer.  There 
are  no  right  or  wrong  answers.  Do  not  spend  too  much  time  on  any  one  statement. 
Remember,  give  the  answer  which  seems  to  describe  how  you  think  or  feel  right 


now. 

Very 
much  so 

Moderately 

so 

Somewhat 

Not 
at  all 

1. 

I  do  not  feel  confident  about  my 
performance  during  qualification. 

□ 

□ 

□ 

□ 

2. 

I  feel  calm. 

□ 

□ 

□ 

□ 

3. 

I  think  my  score  will  be  so  bad  that 
everybody,  including  myself,  will  be 
disappointed. 

□ 

□ 

□ 

a 

4. 

I  feel  tense. 

□ 

□ 

□ 

□ 

5. 

I  am  afraid  that  I  should  have  prepared 
more  for  qualification. 

□ 

□ 

□ 

□ 

6. 

I  feel  at  ease. 

a 

□ 

□ 

□ 

7. 

I  will  not  be  happy  with  my  performance 
during  qualification. 

□ 

□ 

a 

□ 

8. 

I  feel  jittery. 

□ 

□ 

□ 

a 

9. 

I  will  feel  regretful  about  my 
performance  during  qualification. 

□ 

□ 

□ 

□ 

10. 

I  feel  relaxed. 

a 

□ 

□ 

□ 

11.  I  am  concerned  about  what  will  happen  □  □  □ 

if  I  do  poorly. 

Odd  numbers  are  state  worry  items.  Even  numbers  are  state  anxiety  items. 

□ 
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APPENDIX  P 

FIRING  LINE  EXPERIENCE  MEASURE 
(STUDY  2) 


Trait  Version 


In  general,  when  you  are  on  the  firing  line, 
how  often  do  you  ... 

Almost 

always 

Often 

Some¬ 

times 

Almost 

never 

. . .  know  how  the  shot  went  (good  or  bad)  as  soon  as  you  fired 
the  rifle? 

□ 

□ 

□ 

□ 

i..  know  how  to  adiustvouroosition  based  on  vour  prior, 
shot(s)?  -  >. 

a 

Mi 

□ 

'i  s 

.  a  ; 

. . .  know  how  to  adiust  vour  rifle  siohts  based  on  vour  prior 
shot(s)? 

□ 

□ 

□ 

a 

reach  the  “zone”  -  smooth  and  calm  perfornriance,  unaware 
1  of  time  pressure, ^effortless  shooting.  ’  ^ 

□  □ 

<  □ 

□ 

...  get  distracted  mentally  (negative  thoughts  -  “1  can’t  seem  to 
control  myself  or  “There,  1  moved  again”,  and  so  on) 

□ 

□ 

□ 

□ 

|.. ,  get  help  from  the  coaches?®  v  .. 

□ 

a 

. . .  feel  confident  about  your  shooting  performance? 

a 

□ 

a 

□ 

|  . .  feel  anxious  or  "worried  about  your  shooting  performance?'^ 

□ 

□ 

■.V,  ^  '• 

Q  \ 

. . .  use  the  databook?® 

□ 

□ 

a 

□ 

^Dropped  from  analyses  due  to  poor  reliability.  '’Excluded  from  main  analyses  (Study  2)  but  included 
for  longitudinal  analyses  (Appendix  Z). 
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APPENDIX  Q 

PERCEIVED  LEVEL  OF  MARKSMANSHIP  KNOWLEDGE 

(STUDY  2) 


General  questions  about  how  much  you  know  about 
USMC  rifle  marksmanship. 

How  much  do  you  know  about ... 

Very 

much 

Moderate 

amount 

Some 

Not  much 
at  all 

...  the  fundamentals  of  rifle  marksmanship? 

□ 

□ 

□ 

□ 

...  trigger  control? 

□ 

□ 

□ 

□ 

...  breath  control? 

□ 

a 

□ 

□ 

...  the  aiming  process? 

□ 

□ 

□ 

□ 

...  how  trigger  control,  breath  control,  and  the  aiming 
process  all  affect  each  other? 

□ 

□ 

□ 

□ 
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APPENDIX  R 

PERCEIVED  UTILITY  OF  MARKSMANSHIP  KNOWLEDGE 

(STUDY  2) 


Please  indicate  how  much  you  agree  with  each 

Strongly 

Strongly 

statement. 

agree 

Agree 

Disagree  disagree 

A  Marine  can  shoot  well  (qualify  Expert) ... 

without  knowing  what  the  fundamentals  are.  ' 

a 

. . .  without  correctly  applying  the  fundamentals. 

□ 

□ 

□ 

□ 

L  by  shooting  a  lot.^ 

S'.. a 

□ 

Q 

...  by  instinct. 

a 

□ 

□ 

□ 

Knowledge  of  the  fundamentals  (Phase  1  training) ... 

L.  is  important  to  shooting  well.^  '  '.-'Mf- 

□ 

□ 

a  . 

...  is  important  to  improving  shooting  performance. 

□ 

□ 

□ 

□ 

is  used  by  snipers  and  other  top  shooters. 

□ 

□ 

..  □  ■’ 

...  is  useful  to  shooting. 

□ 

□ 

□ 

□ 

‘Dropped  from  analyses  due  to  poor  reliability. 
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APPENDIX  S 
BACKGROUND  SURVEY 
(STUDY  2) 
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Rifle  Marksmanship  Study 
Background  Information 
March  2003 

GENERAL  INFORMATION 


1 .  Age:  _ years 

2.  Gender:  □  Male  □  Female 

3.  Ethnicity:  □  African  American 

□  Asian/Pacific  Islander 

□  Hispanic 


□  Native  American 

□  White 

□  Other _ 


4.  AFQT  (Armed  Forces  Qualifying  Test)  score:  _  □  Don’t  remember 

(sometimes  called  “ASVAB  score”) 

5.  AFQT  category  (I.  II,  IIIA,  NIB,  IVA,  IVB,  IVC,  IV): _  □  Don’t  remember 


6. 


AFQT  line  scores  (if  you  remember): 


□  CL _ 

(clericai,  Administrative, 
Supply  &  Finance) 


□  CO _ 

(combat,  armor,  infantry) 


□  FA _ 

(field  artillery,  cannon,  rocket, 
artillery) 

□  GM _ 

(general  maintenance, 
construction,  utility,  hazmat) 


□  MM _ 

(mechanical  maintenance, 
vehicle  and  aircraft 
maintenance) 

□  OF _ 

(operators  and  food,  food 
service,  drivers  and  missile 
operators) 


□  EL _ 

(electronics  repair,  missile 
repair,  eiectronics  and 
communication) 


□  GT _ 

(general  technical,  special 
and  officer  programs) 


□  Other 


□  Don’t  know 


7.  Highest  levei  of  education  that  you  completed: 

□  4-year  college  or  university  □  High  school 

□  2-year  college  □  Other _ 

□  Trade  or  technical  school 
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CURRENT  SERVICE  INFORMATION 


8.  Number  of  years  in  service:  _ years 


9. 


Rank:  Enlisted  Officer  Warrant  Officer 

□  Private  (E-1)  □  Second  Lieutenant  (0-1)  □  Warrant  Officer  (W-1 ) 

□  Private  First  Class  (E-2)  □  Second  Lieutenant  (0-1 E)  □  Warrant  Officer  2  (W-2) 

□  Lance  Corporal  (E-3)  □  First  Lieutenant  (0-2)  □  Warrant  Officer  3  (W-3) 

□  Corporal  (E-4)  □  First  Lieutenant  (0-2E)  □  Warrant  Officer  4  (W-4) 

□  Sergeant  (E-5)  □  Captain  (0-3)  □  Warrant  Officer  5  (W-5) 

□  Staff  Sergeant  (E-6)  □  Captain  (0-3E) 

□  Gunnery  Sergeant  (E-7)  □  Other _ 

□  Master  Sergeant  (E-8) 

□  First  Sergeant  (E-8) 

□  Other _ 


10.  Unit 


Division 


Wing 


Force  Service 
Support  Group 


Support 


□  Combat  arms  (infantry,  □  Aviation  (squadron  □  Combat  service  support  □  Base/station 

artillery,  armor,  etc.)  or  group)  (supply,  motor  transport, 

maintenance,  etc.) 

□  Combat  support  (engineer,  □  Aviation  command  □  Staff  □  Formal  schools 

communications,  etc.)  and  control 

□  Staff  □  Aviation  ground  □  Marine  security 

support  guard  defense 

□  Staff 


11.  Job  Information: 

Primary  MOS 


Billet  MOS 


Please  indicate  which  courses  you  have  completed: 

12.  Marksmanship  Coach  (MOS  8530) 

13.  Marksmanship  Instructor  (MOS  8531) 

14.  Small  Arms  Weapons  Instructor  (MOS  8532) 

15.  Range  Officer  (MOS  9925) 


Completed 


Did  not  take  or  did  not 
complete 
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RIFLE  MARKSMANSHIP  EXPERIENCE 

Shooting  AS  PART  of  Your  USMC  Duties 

16.  In  general,  how  frequently  do  you  shoot  a  rifle  as  part  of  your  USMC  duties 
(excluding  requalification)? 

□  Never  □  Once  or  twice  □  A  few  times  □  Often  □  Very  often 

17.  Please  estimate  the  number  of  days  you  spend  shooting  a  rifle 

as  part  of  your  USMC  duties  (excluding  requalification): _ 

Recreational  Shooting 

18.  In  general,  how  frequently  do  you  shoot  weapons  outside  of  your  USMC  duties 
(for  hunting,  skeet  shooting,  and  so  on)? 

□  Never  □  Once  or  twice  □  A  few  times  □  Often  □  Very  often 

19.  Please  estimate  the  number  of  days  per  year  you  spend  shooting  a  rifle 

outside  of  your  USMC  duties: _ 

20.  Number  of  years  of  rifle  shooting  experience  prior  to  joining  the  Marines:  _ years 

21 .  How  frequentiy  have  you  shot  competitively? 

□  Never  □  Once  or  twice  □  A  few  times  □  Often  □  Very  often 


In  general,  when  you  are  on  the  firing  line, 
how  often  do  you  ... 

Almost 

always 

Often 

Some¬ 

times 

Almost 

never 

22.  ...  know  how  the  shot  went  (good  or  bad)  as  soon  as  you  fired 
the  rifle? 

□ 

□ 

□ 

□ 

P3.  ...  know  how  to  adjust  vour  position  based  on  vour  prior '  ' 

1  shot(s)?,  ' 

24.  ...  know  how  to  adiust  vour  rifle  siahts  based  on  vour  prior 
shot(s)? 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

25.  ...  reach  the  “zone” -  smooth  and  calm  performance,  unaware 
of  time  pressure,  effortless  shooting,  ^ 

□ 

□ 

0 

26.  ...  get  distracted  mentally  (negative  thoughts  -  “1  can’t  seem  to 
control  myself  or  “There,  1  moved  again”,  and  so  on) 

a 

□ 

□ 

□ 

[27.  ...  get  help  from  the  coaches?  ^ . 

□ 

□ 

□ 

□ 

28.  ...  feel  confident  about  your  shooting  performance? 

□ 

a 

□ 

□ 

[29.  ...  feel  anxious  or  worried  about  your  shooting  performance? 

□ 

□ 

□ 

□ 

30.  ...  use  the  databook? 

a 

a 

□ 

□ 
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CSE  Deliverable 


How  often  do  you  feel  or  think  the  following? 

Almost 

always 

Often 

Some¬ 

times 

Almost 

never 

31 .  Thinking  about  my  score  interferes  with  my  shooting  during 
qualification. 

□ 

□ 

□ 

□ 

[32.  I  freeze  up  during  qualification. 

’  -  □  ■ 

□ 

□ 

a 

33.  During  qualification  I  find  myself  thinking  about  whether  I’ll 
ever  get  through  it. 

□ 

□ 

□ 

□ 

l34.  The  more  I  think  during  qualification,  the  more  confused  I 

I  get.;' " . 

□ 

□  - 

□ 

35.  I  seem  to  defeat  myself  during  qualification. 

□ 

□ 

□ 

□ 

ise.  During  qualification  I  find  myself  thinking  about  the 
|r  consequences  of  failing. 

□ 

□  ; 

□ 

37.  Thoughts  of  doing  poorly  interfere  with  my  concentration 
during  qualification. 

□ 

□ 

□ 

□ 

;38.  During  qualification  I  get  so  nervous  that  I  forget  skills  and 
j  information  I  really  know. 

a 

V' 

QUALIFICATION  INFORMATION 


When  was  your  last ... 

None 

Last 

week 

Other 

39. 

...  training  on  the  fundamentals  of  rifle 
marksmanship  (Phase  I)? 

□ 

□ 

Date  (MM/YY):  _ 

40. 

...  dry-fire  practice  (Phase  I)? 

□ 

□ 

Date  (MMA'Y):  _ 

41. 

...  live-fire  oractice  (Phase  111? 

□ 

□ 

Date  (MMA'Y):  _ 

42. 

...  live-fire  Qualification  trial  (Phase  111? 

□ 

□ 

Date  (MMA'Y): 

Please  list  your  3  most  recent  final  qualification  scores,  starting  with  the  most  recent  # 

one.  Please  estimate  if  you  don’t  remember  your  exact  score. 


43.  Qualification  score: 

(most  recent) 

Date  (MMA'Y): 

44. 

Qualification  score: 

□  Don’t  have  one 

Date  (MMA'Y): 

45. 

Qualification  score: _ 

_  □  Don’t  have  one 

Date  (MMA'Y): 
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APPENDIX  T 


TASK  DIRECTIONS  FOR  SHOT-TO-SHOT  KNOWLEDGE  MAPPING 
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Knowledge  of  Rifle  Marksmanship  Study 
Shot-to-Shot  Analysis 
May  2003 


Task: 

You  will  be  given  pictures  of  a  sequence  of  5  shots.  Your  task  is  to  analyze  the  shot  sequence 
and,  using  the  Knowledge  Mapper,  indicate  reasons  that  might  explain  a  shot’s  placement 
given  the  relationship  among  the  shots  or  the  location  of  the  prior  shot. 

For  example,  given  the  strike  of  round  1,  what  are  some  reasons  that  might  explain  round  2’s 
strike? 

This  task  is  intended  to  approximate  what  a  shooter  has  to  mentally  go  through  on  the  firing 
line.  After  the  shooter  fires  a  shot,  he/she  sees  where  the  round  strikes  the  target.  The  shooter 
then  uses  the  shot  information  to  analyze  the  shot  given  prior  shots  and  help  diagnose  any 
problems.  As  more  rounds  are  fired,  more  information  becomes  available. 

Situation: 

You  can  assume  the  following: 

1 .  All  shots  called  center 

2.  200-yd  slow-fire  sitting  position 

Knowledge  Mapping  Instructions 

Step  1:  First  lay  out  the  nodes  as  shown  below.  Each  node  represents  a  “step”  in  the  firing 
sequence.  You  will  be  given  pictures  on  the  next  page  that  show  the  shot  patterns. 


I^CRESST  Knowledge  Mapper  •  TASK:  Version  t  >  MAP;  tect 

- ,  ilslj 

;.Flle  Edit  View  Help 

^ :P  (Si  o  SI  X  %  ®.|iop%  jrji  j£  =  s 

□ 

i' 

[iround  1  1  i 

(  round  2  |  | 

froundsi,2to3  1  | 

I'tounds  1,2,3  to  4  1  J 
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Step  2:  Connect  the  steps  (nodes).  The  links  represent  the  reasons  that  may  explain  each 
shot,  given  the  previous  shots. 


I^CRESSTKnowledgGMapper  -  TASKliVei^onl  -  MAP:  test 


.=..l.n]2SJ 


siFlle  edit  View  Help  : 

y  #i  it  %  ®  1  ^  r.  ret.  ^ 


jund  1  I 

round  2 


Indicate 
reason(s)  using 
the  iinks 


I  rounds  1,2  to  3  | 

What  are  the  possible  reasons  that  will  explain 
round  2’s  strike  given  round  Ts  strike?  ...  round 
3’s  strike  given  rounds  1  and  2  ...  round  4’s 
strike  given  rounds  1-3  ...  round  5’s  strike  given 
rounds  1-4? 

Indicate  the  reasons  using  one  or  more  links. 

There  may  be  more  than  1  reason/link.  Indicate 
only  the  most  reasonable  reasons/links. 


✓ 


founds  1,2,3  to  4 


✓ 


rounds  1,2, 3, 4  to  5 


Notes: 

1.  Use  the  links  to  indicate  possible  reasons  that  help  explain  a  round’s 
strike  given  prior  shots. 

2.  There  may  be  more  than  1  reason. 

3.  There  may  be  no  problems,  some  problems,  or  many  problems — use 
your  judgment. 

4.  Reasons  may  change  from  shot  to  shot — each  additional  shot  provides 
more  information.  CHANGING  REASONS  FROM  SHOT  TO  SHOT 
IS  OK— THIS  IS  WHAT  HAPPENS  ON  THE  FIRING  LINE.  DON’T 
GO  BACK  AND  CHANGE  ALL  THE  PREVIOUS  REASONS  TO 
MAKE  THEM  “MATCH.” 
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Knowledge  Mapper  Nodes/Steps  and  Links/Reasons 


Nodes/Steps 


round  1 


round  Ito  2 


rounds  1,  2  to  3 


rounds  1,  2,  3  to  4 


rounds  1,  2,  3, 4  to  5 


Reasons/Links 


no  problem(s) 


unable  to  tell 


eye  focused  on  target 


breathing  while  firing 


flinching  (sudden  backward  movement  of 
body)  at  time  of  shot 


bucking  rifle  (movement  of  shoulder  into  the 
rifle  stock)  while  firing 


jerking  trigger  while  firing 


improper  sight  alignment 


improper  sight  picture 


improper  elevation  setting 


improper  windage  setting 


moved  forward  hand  backward  while  firing 


moved  forward  hand  forward  while  firing 


wind  from  left 


wind  from  right 
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CSE  Deliverable 


Round  1 


->  Round  2 


Given  the  strike  of 
round  1 ,  what  are  all 
the  reasonable 
reasons  that  may 
have  caused  round  2 
to  strike  as  shown? 


Rounds  1  and  2 


->  Round  3 


"N  1 

T' . •  ’ 

2y\  I 

Given  the  strike  of 
rounds  1  and  2,  what 
are  all  the  reasonable 
reasons  that  may 
have  caused  round  3 
to  strike  as  shown? 


Rounds  1  thru  3 


->  Round  4 


Given  the  strike  of 
rounds  1  through  3, 
what  are  all  the 
reasonable  reasons 
that  may  have  caused 
round  4  to  strike  as 
shown? 


i 

C 

i . . 

Z 

1 

1  i  -P 
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APPENDIX  U 

TASK  DIRECTIONS  FOR  DATA  BOOK  PROCEDURE  KNOWLEDGE 


MAPPING 
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Knowledge  of  Rifle  Marksmanship  Study 
Data  Book  Procedure 
May  2003 


Task: 

Your  task  is  to  use  the  Knowledge  Mapper  to  show  your  understanding  of  how  to  fill  out  the 
data  book  and  how  to  use  it  as  a  tool  to  improve  your  shooting.  Use  what  you  learned  during 
Phase  I  training  and  your  experience  on  the  firing  line  to  show  the  steps  a  Marine  should  use 
to  fill  out  the  data  book. 

Situation: 

You  can  assume  the  following: 

1 .  200-yd  slow- fire  kneeling  position 

2.  Wind  is  blowing  in  from  the  right  at  half  value 
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Knowledge  Mapping  Instructions 

In  this  task,  the  knowledge  map  will  be  used  to  depict  a  procedure — filling  out  the  data  book. 
The  nodes  in  the  knowledge  map  represent  steps  in  the  procedure,  and  the  links  represent  the 
reasons  why  you  are  going  from  one  step  to  the  next. 


Notes: 

1.  Use  the  links  to  indicate  why  you  are  going  from  one  step  to  another. 

2.  For  reasons  that  are  obvious  or  require  no  explanation,  use  “Next 
Step.”  However,  if  you  can  provide  a  reason,  do  so.  In  general,  “Next 
Step”  is  the  weakest  possible  answer. 

3.  THERE  IS  NO  SINGLE  CORRECT  ANSWER!  THE  TASK  IS 
DESIGNED  TO  ALLOW  FOR  MULTIPLE  CORRECT 
PROCEDURES.  THE  IMPORTANT  PART  IS  THE  QUALITY  OF 
THE  REASONS  BETWEEN  STEPS. 
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Knowledge  Mapper  Nodes/Steps  and  Links/Reasons 


Nodes/Steps 


Links/Reasons 


•  Calculate  and  enter  elevation 
adjustments 

•  Calculate  and  enter  windage 
adjustments 

•  Call  shot 

•  Circle  number  of  clicks  where  wind 
value  and  wind  speed  intersect 

•  Circle  rear  sight  elevation  knob  settings 

•  Determine  direction  of  wind 

•  Determine  value  of  wind 

•  Enter  post-fire  true  zero  FSP  settings 

•  Enter  post-fire  true  zero  RSW  settings 

•  Enter  pre-fire  zero  FSP  settings 
Enter  pre-fire  zero  RSW  settings 

•  Enter  remarks 

•  Establish  true  zero 

•  Establish  zero 

•  Fire  shot 

•  Observe  bullet  strike 

•  Observe  flag  on  range 


•  After  firing 

•  Analyze 

•  As  required 
Before  firing 

•  Compare 

•  Compensate 

•  Make  sight  adjustments 

•  Next  step 

•  Record  information 


Plot  shot 
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Data  Book  Page 
Kneeling  Position 
Slow  Fire,  200  Yards 
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APPENDIX  V 

BASIC  MARKSMANSHIP  KNOWLEDGE  MEASURE 

(STUDY  3) 
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Knowledge  of  Rifle  Marksmanship  Study 
Prior  Knowledge 
May  2003 

Matching 

Directions:  On  the  line  to  the  left  of  each  definition,  write  the  letter  of  the  term  which 
best  matches  the  definition.  Each  term  may  be  used  only  once.  There  are  more  terms 
than  definitions. 


Definition 


Term 


.1.  The  firm  consistent  contact  of  the  cheek  with 
the  weapon's  butt  stock 

.2.  The  distance  between  the  rear  sight  aperture 
and  the  aiming  eye 

.3.  The  body's  skeletal  structure  supporting  the 
rifle's  weight 

,4.  The  point  in  the  breathing  cycle  during  which 
the  body  is  most  relaxed,  allowing  the  sights 
to  settle  at  the  natural  point  of  aim 

.5.  The  skillful  manipulation  of  the  trigger  that 
causes  the  rifle  to  fire  without  disturbing  sight 
alignment  or  sight  picture 

.6.  "V"  formed  between  the  thumb  and  index 

finger  on  the  trigger 

,7.  Continued  application  of  the  fundamentals 
until  the  round  has  exited  the  barrel 


n)  Bone  support 

o)  Eye  relief 

p)  Firing  hand 
placement 

q)  Firm  pistol  grip 

r)  Follow-through 

s)  Muscular  relaxation 

t)  Natural  respiratory 
pause 

u)  Recovery 

v)  Sight  alignment 

w)  Stable  firing  position 

x)  Stock  weld 

y)  Trigger  control 

z)  Zeroing 


8.  Centering  the  clear  tip  of  the  front  sight  post 
both  horizontally  and  vertically  in  the  rear 
sight  aperture 


,9.  The  state  of  tension  required  to  properly 
control  the  rifle 
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Strongly 

agree 

Agree 

Strongly 
Disagree  disagree 

Suppose  a  shooter's  muscles  never  got  fatigued 
(this  is  a  hypothetical  question)  . . . 

10.  ...  it  would  be  OK  to  "muscle"  the  rifle 

□ 

□ 

□ 

□ 

into  position. 

11.  ...  there  would  be  little  need  for  bone 

□ 

□ 

□ 

□ 

support. 

Consistency  is  important  in  shooting  because 

12.  ...  even  very  small  changes  in  body 

□ 

□ 

□ 

□ 

position  from  one  shot  to  the  next  can 
result  in  a  poor  shot. 

13.  ...  it  helps  you  figure  out  what  to  adjust 

□ 

□ 

□ 

□ 

shot  to  shot. 

14.  Given  a  randomly  shaped  target,  which  figure  best  shows  the  correct  aiming 
point? 

d)  Figure  1 

e)  Figure  2 

f)  Figure  3 


Figure  1 


Figure  2 


Figure  3 


Knowledge,  Models  and  Tools  to  Improve  the  Effectiveness  of  Naval  Distance  Learning  285 


Multiple  Choice 

Directions:  Use  this  picture  when  answering  questions  15  and  16.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


15.  Which  best  describes  the  sight  picture/ sight  alignment  shown  in  the  picture 
above? 

e)  Proper  sight  picture,  proper  sight  alignment 

f)  Proper  sight  picture,  improper  sight  alignment 

g)  Improper  sight  picture,  proper  sight  alignment 

h)  Improper  sight  picture,  improper  sight  alignment 


16.  Given  the  above  sight  picture/ sight  alignment,  which  figure  below  best 
shows  where  the  round  would  strike?  Assume  a  properly  zeroed  rifle,  no 
wind  or  weather  effects,  and  a  properly  fired  weapon  at  300  yards. 

e)  Figure  1 

f)  Figure  2 

g)  Figure  3 

h)  Figure  4 


Figure  1 


Figure  2 


Figure  3 


Figure  4 
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Directions:  Use  this  picture  when  answering  questions  17  and  18.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


17.  Which  best  describes  the  sight  picture/  sight  alignment  shown  in  the  picture 
above? 

e)  Proper  sight  picture,  proper  sight  alignment 

f)  Proper  sight  picture,  improper  sight  alignment 

g)  Improper  sight  picture,  proper  sight  alignment 

h)  Improper  sight  picture,  improper  sight  alignment 


18.  Given  the  above  sight  picture/ sight  alignment,  which  figure  below  best 
shows  where  the  round  would  strike?  Assume  a  properly  zeroed  rifle,  no 
wind  or  weather  effects,  and  a  properly  fired  weapon  at  300  yards. 

e)  Figure  1 

f)  Figure  2 

g)  Figures 

h)  Figure  4 
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Directions:  Use  this  picture  when  answering  questions  19  and  20.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


19.  Which  best  describes  the  sight  picture/ sight  alignment  shown  in  the  picture 
above? 

e)  Proper  sight  picture,  proper  sight  alignment 

f)  Proper  sight  picture,  improper  sight  alignment 

g)  Improper  sight  picture,  proper  sight  alignment 

h)  Improper  sight  picture,  improper  sight  alignment 


20.  Given  the  above  sight  picture/ sight  alignment,  which  figure  below  best 
shows  where  the  round  would  strike?  Assume  a  properly  zeroed  rifle,  no 
wind  or  weather  effects,  and  a  properly  fired  weapon  at  300  yards. 

e)  Figure  1 

f)  Figure  2 

g)  Figures 

h)  Figure  4 


Figure  1 


Figure  2 


Figure  3 


Figure  4 
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Directions:  Use  these  pictures  when  answering  questions  21  to  23.  For  each  question, 
select  the  alternative  that  best  answers  the  question. 


Sight  picture  #1 


Sight  picture  #2 


Sight  picture  #3 


Note:  It  may  be  helpful  to  read  all  3  questions  before  answering. 


21.  Which  sight  picture  above  best  reflects  the 
Marine's  eye  relief  shown  in  the  picture  to  the 
left? 

d)  Sight  picture  #1 

e)  Sight  picture  #2 

f)  Sight  picture  #3 


22.  Which  sight  picture  above  best  reflects  the 
Marine's  eye  relief  shown  in  the  picture  to  the 
left? 

d)  Sight  picture  #1 

e)  Sight  picture  #2 

f)  Sight  picture  #3 


23.  Which  sight  picture  above  best  reflects  the 
Marine's  eye  relief  shown  in  the  picture  to  the 
left? 

d)  Sight  picture  #1 

e)  Sight  picture  #2 

f)  Sight  picture  #3 
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Multiple  Choice 

24.  In  the  sitting  position,  which  direction  does  the  rifle  muzzle  move  when  the  rifle 
is  fired  while  inhaling?  Assume  breathing  from  the  chest. 

e)  Up 

f)  Down 

g)  Left 

h)  Right 

25.  Which  of  the  following  safety  rules  best  enforces  muzzle  awareness? 

e)  Treat  every  weapon  as  if  it  were  loaded. 

f)  Never  point  a  weapon  at  anything  you  do  not  intend  to  shoot. 

g)  Keep  your  finger  straight  and  off  the  trigger  until  you  are  ready  to  fire. 

h)  Keep  the  weapon  on  safe  until  you  intend  to  fire. 

26.  WTiich  of  the  following  is  the  proper  rear  sight  elevation  setting  when  firing  at 
the  300-yard  line?  Assume  a  zeroed  rifle. 

e)  5 

f)  5/3 
S)  8/3 
h)  8/3-2 

27.  How  far  will  2  clicks  of  the  rear  sight  windage  move  the  strike  of  the  round  at 
500  yards? 

e)  1  inch 

f)  21/2  inches 

g)  3  3/4  inches 

h)  5  inches 

28.  If  the  Marine's  head  is  not  erect  or  is  creeping  up  on  the  rear  sight,  what  does  this 
indicate? 

a)  Improper  grip 

b)  Improper  stock  weld 

c)  Improper  sling  tension 

d)  Improper  sling  assembly 

29.  Which  reason  best  explains  the  function  of  bone  support? 

a)  To  provide  a  strong  frame  to  absorb  the  rifle  recoil 

b)  To  help  support  the  weight  of  the  rifle 

c)  To  provide  rigid  contact  with  the  ground 

d)  To  allow  the  shooter  to  resist  strong  crosswinds  in  the  field 
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30.  If  a  Marine  is  using  improper  breath  control  during  rifle  firing,  what  is  most 
likely  to  occur? 

a)  The  Marine  will  see  the  front  sight  post  move  up  and  down. 

b)  The  Marine  will  be  consistently  inhaling  and  exhaling  at  the  same  point  in 
time  during  the  respiratory  cycle  for  each  shot. 

c)  The  Marine's  right  elbow  will  be  placed  against  his  or  her  rib  cage. 

d)  The  rifle  butt  will  move  in  the  Marine's  shoulder. 
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Questions  31-32: 

Triangulate  the  shot  group.  Write  the  windage  and  elevation  adjustments  needed  to 
place  the  center  of  the  shot  group  into  the  center  of  the  target.  Specify  the  number  of 
clicks  and  the  direction.  Keep  in  mind  the  rifle  is  set  at  the  initial  sight  setting  (flush 
and  center)  for  each  question. 


UNITS  ARE  IN  CLICKS.  YOU  DO  NOT  HAVE  TO  PERFORM  ANY 
CONVERSIONS. 


1R  1?  e  0  e  n  IS  ?A 
CLICKS 


31.  FSP:  _ clicks,  □  up  □  down 

RSW:  _ clicks,  □  left  □  right 

UNITS  ARE  IN  CLICKS.  YOU  DO  NOT  HAVE  TO 
PERFORM  ANY  CONVERSIONS. 


PLOT 


1?  €  0  e  n  IS 

CLICKS 


32.  FSP:  _ clicks,  □  up  □  down 

RSW:  _ clicks,  □  left  □  right 

UNITS  ARE  IN  CLICKS.  YOU  DO  NOT  HAVE  TO 
PERFORM  ANY  CONVERSIONS. 
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Up  or 
down 

Left  or 
right 

Erratic 

No 

change 

In  which  direction  will  the  natural  point  of  aim  shift 
(right-handed  shooter,  KNEELING)  . . . 

33.  ...  if  the  left  hand  is  moved  along  the 
handguard? 

□ 

□ 

□ 

□ 

• 

34.  ...  if  the  rifle  butt  is  moved  up  or  down  in  the 
shoulder? 

□ 

□ 

□ 

□ 

35.  ...  if  the  left  elbow  is  moved  along  the  knee? 

□ 

□ 

□ 

□ 

• 

36.  ...  if  the  entire  body  is  moved  left  in  relation  to 
the  target? 

In  which  direction  will  the  rifle  muzzle  shift  from  shot 
to  shot . . . 

□ 

□ 

□ 

□ 

• 

37.  ...  if  the  trigger  finger  is  in  contact  with  the 
receiver  or  pistol  grip  while  firing? 

□ 

□ 

□ 

□ 

38.  ...  if  the  shooter  is  holding  his  breath  too  long? 

□ 

□ 

□ 

□ 

• 

39.  ...  if  the  shooter  has  high  muscle  fatigue? 

□ 

□ 

□ 

□ 

40.  ...  if  the  shooter's  position  is  constantly 
changing? 

□ 

□ 

□ 

□ 
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APPENDIX  W 


CLASSROOM  TEST  OF  SCIENTIFIC  REASONING 
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Knowledge  of  Rifle  Marksmanship  Study 
Scientific  Reasoning 
May  2003 


Your  responses  will  be  kept  completely  confidential.  The  results  of  this  survey  will  be  reported  only  for 
this  study  as  a  whole.  Your  particular  responses  will  not  be  identifiable  in  any  way. 


Directions  to  Students: 

This  is  a  test  of  your  ability  to  apply  aspects  of  scientific  and  mathematical  reasoning 
to  analyze  a  situation  to  make  a  prediction  or  solve  a  problem.  Circle  the  letter  next 
to  the  best  answer  for  each  item.  If  you  do  not  fully  understand  what  is  being  asked 
in  an  item,  please  ask  the  test  administrator  for  clarification. 


DO  NOT  OPEN  THIS  BOOKLET  UNTIL  YOU  ARE  TOLD  TO  DO  SO 
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1 .  Suppose  you  are  given  two  balls  of  clay  of  equal  size  and  shape.  The  two  balls  also  weigh  the 
same.  One  of  the  balls  is  flattened  into  a  pancake-shaped  piece.  Which  of  these  statements  is 
correct? 

a.  The  pancake-shaped  piece  weighs  more  than  the  ball 

b.  The  two  pieces  still  weigh  the  same 

c.  The  ball  weighs  more  than  the  pancake-shaped  piece 


2.  because 

a.  the  flattened  piece  covers  a  larger  area. 

b.  the  ball  pushes  down  more  on  one  spot. 

c.  when  something  is  flattened  it  loses  weight. 

d.  clay  has  not  been  added  or  taken  away. 

e.  when  something  is  flattened  it  gains  weight. 


3. 


To  the  right  are  drawings  of  two  cylinders  filled  to  the  same  level  with  water.  The  cylinders  are 
identical  in  size  and  shape. 


4. 


Also  shown  at  the  right  are  two  marbles,  one  glass  and  one 
steel.  The  marbles  are  the  same  size,  but  the  steel  one  is 
much  heavier  than  the  glass  one. 

When  the  glass  marble  is  put  into  Cylinder  1,  it  sinks  to  the 
bottom  and  the  water  level  rises  to  the  6th  mark.  If  we  put  the 
steel  marble  into  Cylinder  2,  then  the  water  will  rise 

a.  to  the  same  level  as  it  did  in  Cylinder  1 

b.  to  a  higher  level  than  it  did  in  Cylinder  1 

c.  to  a  lower  level  than  it  did  in  Cylinder  1 


because 

a.  the  steel  marble  will  sink  faster. 

b.  the  marbles  are  made  of  different  materials. 

c.  the  steel  marble  is  heavier  than  the  glass  marble. 

d.  the  glass  marble  creates  less  pressure. 

e.  the  marbles  are  the  same  size. 


Glass  Marble  Steel  Marble 


Cylinder  1  Cylinder  2 
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5.  To  the  right  are  drawings  of  a  wide  and  a  narrow  cylinder.  The 
cylinders  have  equally  spaced  marks  on  them.  Water  is  poured 
into  the  wide  cylinder  up  to  the  4th  mark  (see  A).  This  water  rises 
to  the  6th  mark  when  poured  into  the  narrow  cylinder  (see  B). 

Both  cylinders  are  emptied  (not  shown)  and  water  is  poured  into 
the  wide  cylinder  up  to  the  6th  mark.  How  high  would  this  water 
rise  if  it  were  poured  into  the  empty  narrow  cylinder? 

a.  to  about  8 

b.  to  about  9 

c.  to  about  10 

d.  to  about  12 

e.  none  of  these  answers  is  correct 


6.  because 

a.  the  answer  cannot  be  determined  with  the  information  given. 

b.  it  went  up  2  more  before,  so  it  will  go  up  2  more  again. 

c.  it  goes  up  3  in  the  narrow  for  every  2  in  the  wide. 

d.  the  second  cylinder  is  narrower. 

e.  one  must  actually  pour  the  water  and  observe  to  find  out. 


7.  Water  is  now  poured  into  the  narrow  cylinder  (described  in  Item  5  above)  up  to  the  1 1th  mark. 
How  high  would  this  water  rise  if  it  were  poured  into  the  empty  wide  cylinder? 

a.  to  about  7  1/2 

b.  to  about  9 

c.  to  about  8 

d.  to  about  7  1/3 

e.  none  of  these  is  correct 


8.  because 

a.  the  ratios  must  stay  the  same. 

b.  one  must  actually  pour  the  water  and  observe  to  find  out. 

c.  the  answer  cannot  be  determined  with  the  information  given. 

d.  it  was  2  less  before  so  it  will  be  2  less  again. 

e.  you  subtract  2  from  the  wide  for  every  3  from  the  narrow. 
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9.  At  the  right  are  drawings  of  three  strings  hanging  from  a  bar.  The  three 
strings  have  metal  weights  attached  to  their  ends.  String  1  and  String  3 
are  the  same  length.  String  2  is  shorter.  A  10  unit  weight  is  attached  to 
the  end  of  String  1.  A  10  unit  weight  is  also  attached  to  the  end  of  String 
2.  A  5  unit  weight  is  attached  to  the  end  of  String  3.  The  strings  (and 
attached  weights)  can  be  swung  back  and  forth  and  the  time  it  takes  to 
make  a  swing  can  be  timed. 

Suppose  you  want  to  find  out  whether  the  length  of  the  string  has  an  effect 
on  the  time  it  takes  to  swing  back  and  forth.  Which  string(s)  would  you 
use  to  find  out? 

a.  only  one  string 

b.  all  three  strings 

c.  2  and  3 

d.  1  and  3 

e.  1  and  2 


1  2  3 


10.  because 

a.  you  must  use  the  longest  strings. 

b.  you  must  compare  strings  with  both  light  and  heavy  weights. 

c.  only  the  lengths  differ. 

d.  to  make  all  possible  comparisons. 

e.  the  weights  differ. 
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1 1 .  Twenty  fruit  flies  are  placed  in  each  of  four  glass  tubes.  The  tubes  are  sealed.  Tubes  I  and  II 
are  partially  covered  with  black  paper;  Tubes  III  and  IV  are  not  covered.  The  tubes  are  placed 
as  shown.  Then  they  are  exposed  to  red  light  for  five  minutes.  The  number  of  flies  in  the 
uncovered  part  of  each  tube  is  shown  in  the  drawing. 

Red  Light 

i  i  i  1  i  1  i 


I  m 


18 

19 

IV 

1 

QjDW 

1 

d 

10  )  10  ) 

t  t  T  t  t  t  t 

Red  Light 


This  experiment  shows  that  flies  respond  to  (respond  means  move  to  or  away  from): 

a.  red  light  but  not  gravity 

b.  gravity  but  not  red  light 

c.  both  red  light  and  gravity 

d.  neither  red  light  nor  gravity 


12.  because 

a.  most  flies  are  in  the  upper  end  of  Tube  III  but  spread  about  evenly  in  Tube  II. 

b.  most  flies  did  not  go  to  the  bottom  of  Tubes  I  and  III. 

c.  the  flies  need  light  to  see  and  must  fly  against  gravity. 

d.  the  majority  of  flies  are  in  the  upper  ends  and  in  the  lighted  ends  of  the  tubes. 

e.  some  flies  are  in  both  ends  of  each  tube. 
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13.  In  a  second  experiment,  a  different  kind  of  fly  and  blue  light  was  used.  The  results  are  shown 
in  the  drawing. 
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Blue  Light 


These  data  show  that  these  flies  respond  to  (respond  means  move  to  or  away  from): 

a.  blue  light  but  not  gravity 

b.  gravity  but  not  blue  light 

c.  both  blue  light  and  gravity 

d.  neither  blue  light  nor  gravity 


14.  because 

a.  some  flies  are  in  both  ends  of  each  tube. 

b.  the  flies  need  light  to  see  and  must  fly  against  gravity. 

c.  the  flies  are  spread  about  evenly  in  Tube  IV  and  in  the  upper  end  of  Tube  III. 

d.  most  flies  are  in  the  lighted  end  of  Tube  II  but  do  not  go  down  in  Tubes  I  and  III. 

e.  most  flies  are  in  the  upper  end  of  Tube  I  and  the  lighted  end  of  Tube  II. 


15.  Six  square  pieces  of  wood  are  put  into  a  cloth  bag  and  mixed  about.  The  six 
pieces  are  identical  in  size  and  shape,  however,  three  pieces  are  red  and  three 
are  yellow.  Suppose  someone  reaches  into  the  bag  (without  looking)  and  pulls 
out  one  piece.  What  are  the  chances  that  the  piece  is  red? 


a.  1  chance  out  of  6 

b.  1  chance  out  of  3 

c.  1  chance  out  of  2 

d.  1  chance  out  of  1 

e.  cannot  be  determined 


16.  because 

a.  3  out  of  6  pieces  are  red. 

b.  there  is  not  way  to  tell  which  piece  will  be  picked 

c.  only  1  piece  of  the  6  in  the  bag  is  picked. 

d.  all  6  pieces  are  identical  in  size  and  shape. 

e.  only  1  red  piece  can  be  picked  out  of  the  3  red  pieces. 
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17.  Three  red  square  pieces  of  wood,  four  yellow  square  pieces,  and  five  blue  square  pieces  are 
put  into  a  cloth  bag.  Four  red  round  pieces,  two  yellow  round  pieces,  and  three  blue  round 
pieces  are  also  put  into  the  bag.  All  the  pieces  are  then  mixed  about.  Suppose  someone 
reaches  into  the  bag  (without  looking  and  without  feeling  for  a  particular  shape  piece)  and  pulls 
out  one  piece. 


What  are  the  chances  that  the  piece  is  a  red  round  or  blue  round  piece? 

a.  cannot  be  determined 

b.  1  chance  out  of  3 

c.  1  chance  out  of  21 

d.  15  chances  out  of  21 

e.  1  chance  out  of  2 


18.  because 

a.  1  of  the  2  shapes  is  round. 

b.  1 5  of  the  21  pieces  are  red  or  blue. 

c.  there  is  no  way  to  tell  which  piece  will  be  picked. 

d.  only  1  of  the  21  pieces  is  picked  out  of  the  bag. 

e.  1  of  every  3  pieces  is  a  red  or  blue  round  piece. 
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19.  Farmer  Brown  was  observing  the  mice  that  live  in  his  field.  He  discovered  that  all  of  them  were 
either  fat  or  thin.  Also,  all  of  them  had  either  black  tails  or  white  tails.  This  made  him  wonder  if 
there  might  be  a  link  between  the  size  of  the  mice  and  the  color  of  their  tails.  So  he  captured 
all  of  the  mice  in  one  part  of  his  field  and  observed  them.  Below  are  the  mice  that  he  captured. 


Do  you  think  there  is  a  iink  between  the  size  of  the  mice  and  the  coior  of  their  taiis? 

a.  appears  to  be  a  link 

b.  appears  not  to  be  a  link 

c.  cannot  make  a  reasonable  guess 


20.  because 

a.  there  are  some  of  each  kind  of  mouse. 

b.  there  may  be  a  genetic  link  between  mouse  size  and  tail  color. 

c.  there  were  not  enough  mice  captured. 

d.  most  of  the  fat  mice  have  black  tails  while  most  of  the  thin  mice  have  white  tails. 

e.  as  the  mice  grew  fatter,  their  tails  became  darker. 
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21 .  The  figure  below  at  the  left  shows  a  drinking  glass  and  a  burning  birthday  candle  stuck  in  a 
small  piece  of  clay  standing  in  a  pan  of  water.  When  the  glass  is  turned  upside  down,  put  over 
the  candle,  and  placed  in  the  water,  the  candle  quickly  goes  out  and  water  rushes  up  into  the 


This  observation  raises  an  interesting  question;  Why  does  the  water  rush  up  into  the  glass? 

Here  is  a  possible  explanation.  The  flame  converts  oxygen  into  carbon  dioxide.  Because 
oxygen  does  not  dissolve  rapidly  into  water  but  carbon  dioxide  does,  the  newly-formed  carbon 
dioxide  dissolves  rapidly  into  the  water,  lowering  the  air  pressure  inside  the  glass. 

Suppose  you  have  the  materials  mentioned  above  plus  some  matches  and  some  dry  ice  (dry 
ice  is  frozen  carbon  dioxide).  Using  some  or  all  of  the  materials,  how  could  you  test  this 
possible  explanation? 

a.  Saturate  the  water  with  carbon  dioxide  and  redo  the  experiment  noting  the  amount  of  water 
rise. 

b.  The  water  rises  because  oxygen  is  consumed,  so  redo  the  experiment  in  exactly  the  same 
way  to  show  water  rise  due  to  oxygen  loss. 

c.  Conduct  a  controlled  experiment  varying  only  the  number  of  candles  to  see  if  that  makes  a 
difference. 

d.  Suction  is  responsible  for  water  rise,  so  put  a  balloon  over  the  top  of  an  open-ended 
cylinder  and  place  the  cylinder  over  the  burning  candle. 

e.  Redo  the  experiment,  but  make  sure  it  is  controlled  by  holding  all  independent  variables 
constant:  then  measure  the  amount  of  water  rise. 


22.  What  result  of  your  test  (mentioned  in  #21  above)  would  show  that  your  explanation  is  probably 
wrong? 

a.  The  water  rises  the  same  as  it  did  before. 

b.  The  water  rises  less  than  it  did  before. 

c.  The  balloon  expands  out. 

d.  The  balloon  is  sucked  in. 
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23.  A  student  put  a  drop  of  blood  on  a  microscope  slide  and  then  looked  at  the  blood  under  a 

microscope.  As  you  can  see  in  the  diagram  below,  the  magnified  red  blood  cells  look  like  little 
round  balls.  After  adding  a  few  drops  of  salt  water  to  the  drop  of  blood,  the  student  noticed  that 
the  cells  appeared  to  become  smaller. 


&  Q  ^ 

Q,  03  <2 


Magnified  Red  Blood  Cells 


After  Adding  Salt  Water 


This  observation  raises  an  interesting  question:  Why  do  the  red  blood  cells  appear  smaller? 

Here  are  two  possible  explanations:  1.  Salt  ions  (Na+  and  CI-)  push  on  the  cell  membranes  and 
make  the  cells  appear  smaller.  2.  Water  molecules  are  attracted  to  the  salt  ions  so  the  water 
molecules  move  out  of  the  cells  and  leave  the  cells  smaller. 

To  test  these  explanations,  the  student  used  some  salt  water,  a  very  accurate  weighing  device, 
and  some  water-filled  plastic  bags,  and  assumed  the  plastic  behaves  just  like  red-blood-cell 
membranes.  The  experiment  involved  carefully  weighing  a  water-filled  bag,  then  placing  the 
bag  in  a  salt  solution  for  ten  minutes,  and  then  reweighing  the  bag  after  it  was  taken  out  of  the 
solution. 


What  result  of  the  experiment  would  best  show  that  explanation  I  is  probably  wrong? 

a.  the  bag  loses  weight 

b.  the  bag  weighs  the  same 

c.  the  bag  appears  smaller 


24.  What  result  of  the  experiment  would  best  show  that  explanation  II  is  probably  wrong? 

a.  the  bag  loses  weight 

b.  the  bag  weighs  the  same 

c.  the  bag  appears  smaller 
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Classroom  Test  of  Scientific  Reasoning  Key 


24 


b 


Knowledge,  Models  and  Tools  to  Improve  the  Effectiveness  of  Naval  Distance  Learning  307 


APPENDIX  X 

ESP  SCORING  RUBRIC  (STUDY  3) 


Scenerio 

Correct 

1 

prone 

all  position  elements  correct 

2 

prone 

contains  incorrect  position  elements 

3 

sitting 

all  position  elements  correct 

4 

sitting 

contains  incorrect  position  elements 

5 

kneeling 

all  position  elements  correct 

6 

kneeling 

contains  incorrect  position  elements 

7 

standing 

all  position  elements  correct 

8 

standing 

contains  incorrect  position  elements 

See-  fwd 

nerio  hand 

firing 

hand 

elbow 

eye  leg/feet 

relief  placment 

rifle 

butt 

leg 

foot 

body 

2 

1 

1 

n/ a 

n/a 

1 

4 

1 

1 

1 

n/a 

n/a 

n/a 

6 

n/a 

1 

1 

1 

n/a 

8 

1 

1 

n/a 

n/a 

1 

Note.  A  "1"  indicates  an  incorrect  position  element,  "n/a"  indicates  position  element  not  part  of 
scenario. 
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APPENDIX  Y 

ESTIMATING  CHANGE  TRAJECTORIES  OF  RECORD-FIRE  PERFORMANCE 

This  study  focused  on  the  53  second  LTs  in  Study  3  who  went  through  entry- 
level  marksmanship  training.  The  training  was  composed  of  classroom  instruction 
on  rifle  marksmanship  in  the  beginning,  live-fire  practice  for  four  days,  and  a 
qualifying  trial  on  the  last  day.  For  each  of  the  live-fire  practices  and  the  qualifying 
trial,  all  53  2nd  LTs  completed  surveys  which  asked  about  their  record-fire  scores  for 
each  day  and  about  how  they  thought  or  felt  about  their  own  record-fire 
performance  for  that  day. 

This  section  is  concerned  with  estimating  change  trajectories  of  Marines' 
record-fire  performance  over  5  days,  via  applying  growth  modeling  techniques  to 
the  survey  data.  Growth  modeling  techniques  are  a  well-established  method  and 
have  been  extremely  useful  in  various  research  areas  when  there  is  a  moderate 
number  of  repeated  observations  for  a  moderate  number  of  subjects  (e.g.,  in 
education,  Raudenbush  &  Bryk,  2002).  Estimation  of  the  change  trajectories  may 
suggest  how  2nd  LTs  with  no  prior  knowledge  change  in  their  shooting 
performance  over  a  short  but  intensive  training  period  (e.g.,  in  this  study,  2  days  of 
classroom  instruction  and  5  days  of  live-fire  practice  or  qualifying  trial).  Growth 
modeling  techniques  can  be  used  to  address  the  following  questions: 

1.  What  is  the  expected  level  of  record-fire  performance  for  a  typical  Marine  in 
the  beginning  of  the  study  (i.e.,  initial  status)? 

2.  How  fast  does  a  typical  Marine  change  his  shooting  performance  (i.e., 
growth  rate)? 

3.  What  is  the  individual  variation  around  the  expected  level  (1)  or  the 
expected  change  rate  (2)? 

4.  Which  characteristics  of  a  Marine  relate  to  the  level  of  shooting 
performance? 

5.  Which  characteristics  of  a  Marine  relate  to  the  change  rate  of  shooting 
performance? 

Questions  1  and  2  concern  the  estimation  of  the  change  trajectories  of  a  typical 
2nd  LT.  Question  3  concerns  individual  variations  around  the  change  trajectories. 
Questions  4  and  5  concern  the  correlates  of  the  level  or  the  change.  Questions  4  and 
5  are  reserved  in  this  section  for  future  analyses. 
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Before  presenting  the  statistical  models  and  the  results,  it  would  be  helpful  to 
discuss  two  features  of  the  data.  First,  there  were  five  repeated  measures  — the 
surveys  of  five  consecutive  days.  However,  the  observations  made  on  the  first  day 
have  not  been  used  in  the  analysis.  Because  there  were  too  many  missing 
observations  on  the  first  day,  it  was  hard  to  assume  that  the  mechanisms  underlying 
the  missing  data  would  be  "ignorable"  (Little  &  Rubin,  1987).  When  missing-data 
mechanisms  are  "nonignorable,"  the  data  analysis  has  to  deal  with  it,  since  many 
statistical  software  such  as  SAS  Proc  Mixed  assumes  that  data  are  missing  at  random 
(MAR),  which  belongs  to  an  ignorable  missing  mechanism.  Although  there  are  more 
advanced  ways  to  deal  with  nonignorable  missing  data  (Hedeker  &  Gibbons,  1997; 
Little,  1995;  Little  &  Rubin,  1987,  chap.  11),  in  this  section,  we  analyze  the  data 
excluding  the  observations  for  the  first  occasion. 

Table  69  presents  descriptive  statistics  of  repeated  measures  on  each  day:  The 
first  panel  shows  all  available  observations,  and  the  second  panel  shows  the 
repeated  observations  for  only  the  2nd  LTs  who  reported  their  shooting  scores  on 
the  first  day  (Dayl).  From  the  first  panel,  it  is  clear  that  no  low-performing  people 
reported  data  on  the  first  day,  as  the  minimum  score  for  the  first  day  is  193,  in 
comparison  to  170  and  175  for  the  next  three  days.  It  is  further  confirmed  in  the 
second  panel  of  the  table.  The  minimum  scores  for  the  participants  who  reported  on 
the  first  day  were  186-190  for  the  later  days,  while  for  all  available  participants  the 
minimum  scores  were  170-190  for  the  later  days.  Thus,  one  may  conclude  that  the 
missing-data  mechanism  on  the  first  day  is  nonignorable  (Little  &  Rubin,  1987). 
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Table  69 


Descriptive  Statistics  of  Repeated  Measures  of  Record- 
Fire  Performance 


n 

M 

SD 

Min. 

Max. 

Dayl 

10 

206.50 

9.76 

193 

222 

Day2 

32 

213.37 

17.43 

170 

237 

Day3 

40 

216.17 

14.28 

175 

235 

Day4 

47 

215.72 

15.83 

175 

240 

Day5 

53 

214.45 

14.46 

190 

240 

Dayl 

10 

206.50 

9.76 

193 

222 

Day2 

9 

214.66 

17.09 

186 

237 

Day3 

10 

218.30 

14.87 

186 

234 

Day4 

10 

214.80 

15.59 

188 

234 

Day5 

10 

217.40 

14.22 

190 

231 

For  the  second  special  feature  of  the  data,  the  observations  on  the  last  day  (i.e., 
the  fifth  day)  are  substituted  by  the  record-fire  performance  of  the  qualifying  trial. 
The  last-day  observations  in  the  post-fire  surveys  are  supposed  to  be  equivalent  to 
the  record-fire  scores  of  the  qualifying  exam.  The  difference  is  that  the  observations 
in  the  post-fire  surveys  were  self-reported,  while  the  qualifying  record-fire  scores 
are  official  scores  on  record.  Given  that  the  qualifying  record-fire  scores  would  be 
more  reliable  than  self-reported  scores,  we  used  qualifying  record-fire  scores  instead 
of  self-reported  scores  in  the  survey. 

As  expected,  the  two  scores  are  not  equivalent  for  many  people.  Figure  17 
presents  a  stem-and-leaf  plot  and  a  boxplot  of  the  difference  values  between  self- 
reported  and  qualifying  record-fire  scores  (i.e.,  difference  =  self-reported  scores  - 
qualifying  scores).  The  difference  values  mostly  range  between  -4  and  4;  within  this 
range  the  scores  make  a  bell-shaped  distribution.  It  seems  to  be  random  errors  of 
self-reported  scores  due  to  forgetfulness,  for  example.  There  are  also  three  outliers 
with  values  of  7  and  two  13s. 
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Figure  17.  A  stem-and-leaf  plot  and  a  boxplot  of  the  difference  values,  which  is  calculated  by 
subtracting  qualifying  exam  scores  from  self-reported  scores. 


In  sum,  the  data  have  four  repeated  measures  of  shooting  performance  from 
the  second  live-fire  day  to  the  qualifying  day  (i.e.,  the  fifth  day),  with  the  measures 
of  the  last  occasion  being  replaced  by  the  qualifying  record-fire  scores. 

The  following  statistical  model  is  posed  for  the  data: 


Yti  =  noi  +  nii  (Dayii  -  Day..)  +  nai  (Dayu  -  Day..)2  +  cti  r,i  ~  N(0,  ) ,  (1) 

where  Yti  is  a  measure  of  knowledge  map  of  a  Marine  i  at  occasion  t,  t=l,  ...  4;  and 
Day  is  the  number  of  days  since  the  first  live-fire  practice  day  (i.e.,  takes  on  values  of 
2,  3,  4,  and  5). 


^Oi  “  Poo  +  Toi 


Toi  ~  N(0,  Too), 


—  Pio  +  Tii 


r,i~N(0,  Tn), 


(2) 


^li  -  P20- 

Equation  (1)  is  a  level-1  model  that  estimates  the  change  trajectory  of  a  2nd  LT 
i,  which  is  also  referred  to  as  a  wi thin-individual  model.  The  equation  includes  a 
second  order  polynomial  of  the  time-clocking  variable  (i.e..  Day),  to  capture  a 
curvature  in  the  change  pattern.  The  existence  of  a  curvature  in  the  change  pattern 
implies  that  the  instantaneous  change  rate  (i.e.,  a  change  rate  at  a  given  time  point) 
is  not  constant  over  time.  Raudenbush  and  Bryk  (2002)  term  this  kind  of  change 
pattern  a  "quadratic  growth  model,"  while  Singer  and  Willet  (2003)  term  this  a 
"quadratic  change  trajectory."  In  comparison  to  a  linear  change  trajectory,  a 
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quadratic  change  trajectory  fitted  slightly  better  (x^df=i  =  3.4,  p  value  =  .065;  and  AlC 
is  1363.3  for  the  quadratic  change  and  1364.7  for  the  linear  change). 

The  time-clocking  variable  (i.e..  Day)  is  centered  at  the  average  of  the  Day 
variable  (i.e.,  3.5).  By  virtue  of  this  centering,  the  parameter  noi  represents  the  level  of 
record-fire  performance  of  a  2nd  LT  i  in  the  middle  of  the  training  period;  and  the 
parameter  nii  represents  the  instantaneous  change  rate  of  a  2nd  LT  i  in  the  middle  of 
the  training  period.  The  parameter  nn  captures  the  curvature  parameter  that  is  not 
contingent  on  the  centering  or  the  acceleration  in  the  change  pattern.  While  the 
meaning  of  the  other  parameters  is  contingent  upon  the  centered  value  (e.g.,  average 
of  the  Day  variable,  which  is  3.5),  the  meaning  of  the  curvature  parameter  is  not. 
Centering  at  the  average  of  the  time-clocking  variable  is  desirable  in  higher  order 
polynomial  models,  as  it  decreases  a  high  correlation  between  the  time  variables 
(e.g.,  in  this  analysis,  the  first  and  the  second  polynomial  of  the  Day  variable). 

Equation  (2)  is  a  level-2  model  that  is  also  referred  to  as  a  between-individual 
model.  The  performance  level  and  the  change  rate  in  the  middle  of  the  training 
period  are  posed  to  be  random  (i.e.,  varying  across  Marines).  The  curvature 
parameter  is  posed  to  be  fixed.  In  comparison  to  the  random  curvature,  the  fixed 
curvature  model  fitted  better  to  the  data  (x^df=3  =  1.3,  p  value  =  .73;  and  AIC  is  1363.3 
for  the  fixed  curvature  and  1368.0  for  the  random  curvature). 

Table  70  presents  the  results  of  the  analysis.  The  upper  panel  presents  fixed 
effects,  which  captures  an  average  change  trajectory  for  a  typical  2nd  LT.  In  the 
middle  of  the  live-fire  training  period,  a  typical  Marine  is  expected  to  score  about 
215  points.  In  the  middle  of  the  period,  a  typical  Marine  increases  his  score  1.4 
points  per  day.  The  curvature  is  minus  1.26  points,  i.e.,  the  deceleration  is  1.26.  The 
Marines  tend  to  increase  their  live-fire  performance  at  a  faster  rate  in  the  beginning 
and  at  a  slower  rate  at  the  end  of  the  training  period. 
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Table  70 

Results  of  Fitting  the  Above  Growth  Model  (N  =  53) 

Parameter 

Estimate 

Fixed  effects 

Mean  shooting  score  in  the  middle  of  the  training  period 

o 

o 

215.12***  (2.03) 

Mean  change  rate  in  the  middle  of  the  training  period 

Pio 

1.41*  (0.84) 

Mean  curvature 

(320 

-1.26*  (0.67) 

Variance  components 

Level -1:  Within-person 

o^ 

20.78*** 

Level-2:  Within-individual 

o^ 

70  27*** 

In  status  in  the  middle  of  the  training  period 

Too 

156.26*** 

In  change  rate  in  the  middle  of  the  training  period 

Tn 

13.30** 

Note:  The  model  is  estimated  using  SAS  Proc  Mixed.  The  covariance  component  was  estimated  but 
not  displayed  for  the  simplicity  of  presentation. 

*  p  <  .05.  **  p  <  .01.  ***  p  <  .001. 

The  lower  panel  presents  variance  components.  The  level-2  variance 
components  estimate  between-individual  variation  around  the  corresponding  fixed 
effects.  Second  LTs  are  significantly  different  in  their  status  and  change  rate  in  the 
middle  of  the  training  period.  Figure  18  displays  the  estimated  change  trajectories  of 
three  hypothetical  2nd  LTs.  The  middle  trajectory  is  the  average  trajectory;  the 
increasing  trajectory  is  the  trajectory  of  a  Marine  who  has  the  change  rate  of  1 
standard  deviation  above  the  mean  change  rate;  and  the  decreasing  trajectory  is  that 
of  a  Marine  who  has  the  change  rate  of  1  standard  deviation  below  the  mean.  One 
may  see  the  wide  variation  across  individuals  in  the  change  pattern.  On  average,  a 
2nd  LT  tended  to  roughly  increase  over  time,  but  faster  in  the  beginning  and  slower 
toward  the  end,  scoring  around  215  in  the  middle  of  the  training.  However,  there  is 
a  wide  range  in  the  average  performance  and  the  change  rate  across  2nd  LTs.  Some 
increase  more  rapidly,  and  others  even  decrease  over  time.  The  great  range  of 
variation  between  individuals  may  be  understood  when  one  considers  that  the  2nd 
LTs  had  no  prior  experience  with  marksmanship.  According  to  the  phases-of-skill- 
learning  theory,  when  one  begins  to  learn  skills  such  as  marksmanship,  performance 
tends  to  be  inconsistent  and  fallible. 
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Figure  18.  Three  estimated  change  trajectories:  the  average,  one  standard 
deviation  below  the  average  change  rate,  and  one  standard  deviation  above  the 
average  change  rate. 
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APPENDIX  Z 

LONGITUDINAL  ANALYSES  OF  KNOWLEDGE  MAPPING 

Repeated  measures  of  knowledge  maps  in  a  short  span  of  time  could  help  us 
understand  the  process  of  learning.  In  settings  where  learning  is  actively  aided  by 
intended  interventions,  repeated  measures  over  a  short  span  of  time  could  provide 
insight  on  how  people  have  changed  their  knowledge  maps  in  response  to  the 
interventions. 

This  study's  sample  is  the  2nd  LTs  going  through  entry-level  marksmanship 
training  in  Study  3  (N  =  53).  In  this  study,  knowledge  maps  were  administered  on 
six  occasions:  before  and  after  classroom  instruction,  two  times  during  live-fire 
practice,  and  two  times  after  qualification.  To  investigate  how  Marines  change  their 
knowledge  maps  in  response  to  classroom  instruction  or  live-fire  experience,  we 
employed  the  statistical  tool  of  growth  modeling  techniques. 

Growth  modeling  techniques  are  a  well-established  method  in  statistics  and 
have  been  extremely  useful  in  various  research  areas  when  there  is  a  moderate 
number  of  repeated  observations  for  a  moderate  number  of  subjects  (e.g.,  in 
education,  Raudenbush  &  Bryk,  2002).  However,  the  techniques  have  not  yet  been 
employed  in  studying  knowledge  maps.  Growth  modeling  techniques  would  help 
provide  pertinent  insights  to  the  study  of  knowledge  maps  by  examining  the 
following  questions: 

1.  What  is  the  expected  level  for  a  typical  Marine's  knowledge  map  at  the 
beginning  of  the  study  (i.e.,  initial  status)? 

2.  How  fast  does  a  typical  Marine  change  his  knowledge  maps  to  be  more  like 
those  of  experts  (i.e.,  growth  rate)? 

3.  How  much  is  the  individual  variation  around  the  expected  initial  status  (1) 
or  the  expected  growth  rate  (2)? 

4.  Which  characteristics  of  a  Marine  relate  to  the  initial  level  of  knowledge 
maps? 

5.  Which  characteristics  of  a  Marine  relate  to  the  growth  rate  of  knowledge 
maps? 

Questions  1-3  quantify  growth  patterns  of  Marines  in  their  knowledge  maps. 
As  the  53  second  LTs  had  no  prior  experience  with  rifles,  their  knowledge  maps 
provided  data  to  study  the  growth  patterns  in  knowledge  maps  of  beginner  learners 
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when  they  are  exposed  to  intensive  classroom  instruction  followed  by  live-fire 
practice. 

However,  these  questions  may  be  viewed  from  a  different  perspective.  If  most 
Marines  change  their  knowledge  maps  to  be  more  like  those  of  experts  in  response 
to  classroom  instruction  or  live-fire  experience  at  a  significant  rate,  it  could  provide 
evidence  that  knowledge  maps  are  sensitive  to  classroom  instruction  or  live-fire 
experience.  The  questions  for  this  strand  of  inquiry  are: 

1.  How  sensitive  are  knowledge  maps  to  classroom  instruction? 

2.  How  sensitive  are  knowledge  maps  to  the  live-fire  practice  sessions  that 
follow  classroom  instruction? 

To  examine  the  questions  addressed  above  (Questions  4  and  5  are  reserved  in 
this  section  for  future  analyses),  a  statistical  model  is  posed  as  follows: 


Yti  =  noi  +  nii  Timel  +  n2i  Time?  +  et,  rti  ~  N(0,  ) ,  (1) 

where  Yti  is  a  measure  of  the  knowledge  map  of  a  Marine  i  at  occasion  t,  t=l,  ...  6; 
Timel  is  the  number  of  days  since  the  first  measure  during  classroom  instruction; 
and  Time2  is  the  number  of  days  since  the  end  of  classroom  instruction  during  live- 
fire  practice,  qualification,  and  after  qualification. 

noi  =  Poo  +  roi  roi  -  N(0,  Too), 

nii  =  P)o  +  rii  rii  ~  N(0,  xn),  (2) 

n2i  =  P20  +  r2i  r2i  ~  N(0,  T22). 

Equation  (1)  is  a  level-1  model  that  deals  with  repeated  measures  within  a  2nd 
LT  i;  and  equations  that  follow  are  a  leveI-2  model  that  examines  the  growth  pattern 
parameters  at  level  1  as  an  outcome.  Thus,  the  level-1  model  can  be  viewed  as  a 
within-individual  model  while  the  level-2  model  can  be  viewed  as  a  between- 
individual  model. 

The  key  parameters  are  Pio  and  P20,  which  are  the  expected  rate  of  change 
during  classroom  instruction  and  the  expected  rate  of  change  during  live-fire 
experience,  respectively.  The  variance  components,  th  and  T22,  capture  individual 
variations  around  the  expected  rates,  Pio  and  P20,  respectively. 

Note  that  in  this  particular  study,  there  are  two  time  clocking  variables,  Timel 
and  Time2  (Equation  1).  One  clocks  time  during  the  classroom  instruction  and  the 
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other  clocks  time  during  the  live-fire  experience  that  followed  the  classroom 
instruction.  This  is  based  on  the  strong  rationale  that  the  Marines  will  change  at  a 
faster  rate  during  the  classroom  instruction  period  than  the  other  period,  because  it 
intensively  teaches  the  Marines  so  that  they  can  construct  knowledge  maps  closer  to 
those  of  experts.  Thus  different  slopes  (i.e.,  rates  of  change)  for  the  different  periods 
are  initially  hypothesized  in  the  model,  using  two  time  clocking  variables  Timel  and 
Time2.  Raudenbush  and  Bryk  (2002)  refer  to  this  parameterization  as  piecewise 
growth  models,  and  Singer  and  Willet  (2003)  term  this  as  discontinuous  individual 
change. 

Table  71  presents  the  results.  A  typical  2nd  LT  tends  to  score  12.99  in  the 
knowledge  map  measures  before  any  instruction  or  any  live-fire  experience.  During 
the  classroom  instruction,  a  2nd  LT  increases  1.76  points  a  day  on  average;  and 
during  the  following  live-fire  experience,  he  tends  to  increase  0.37  points  a  day.  That 
is,  on  average,  a  2nd  LT  changes  his  knowledge  maps  at  a  significantly  fast  rate  into 
those  of  experts  in  response  to  the  classroom  instruction,  and  at  a  significant  but 
slower  rate  in  response  to  the  live-fire  experience  that  followed  the  classroom 
instruction.  It  is  notable  that  the  change  rate  during  the  first  period  is  more  than  4 
times  the  change  rate  during  the  second  period. 

Table  71 


Results  of  Fitting  the  Above  Growth  Model  (n  =  53) 


Parameter 

Estimate 

Fixed  effects 

Expected  initial  status 

Poo 

12.99***  (1.39) 

Expected  rate  of  change  during  classroom  instruction 

pio 

1.76**  (0.59) 

Rate  of  change  during  live-fire  experience 

P20 

0.37*  (0.14) 

Variance  Components 

Level-1:  Within-person 

o^ 

20.78*** 

Level-2:  In  initial  status 

Too 

79.15*** 

In  rate  of  change  during  classroom  instruction 

Til 

9.11** 

In  rate  of  change  during  live-fire  experience 

T22 

0.75** 

Note.  The  model  is  estimated  using  SAS  Proc  Mixed.  All  relevant  covariance  components 
were  estimated  but  not  displayed  for  the  simplicity  of  presentation. 

*p<  .05.  **p<  .01.  ***  p  <  .001. 
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In  addition,  there  is  a  significant  variation  around  all  growth  parameters,  initial 
status,  the  rate  of  change  during  classroom  instruction,  and  the  rate  of  change 
during  live-fire  experience.  This  suggests  that  2nd  LTs  show  individual  differences 
in  where  they  begin,  how  fast  they  grow  in  response  to  classroom  instruction,  and 
how  fast  they  grow  in  response  to  live-fire  experience.  For  example,  a  2nd  LT  who  is 
one  standard  deviation  above  the  rate  of  change  during  classroom  instruction  tends 
to  increase  4.78  points  per  day  during  the  classroom  instruction,  while  a  2nd  LT  who 
is  one  standard  deviation  below  decreases  1.26  points  per  day  during  the  classroom 
instruction.  Figure  19  displays  growth  patterns  of  three  hypothetical  2nd  LTs  with 
an  average,  one  standard  deviation  above,  and  one  standard  deviation  below  rate  of 
change  during  classroom  instruction,  other  growth  parameters  (i.e.,  initial  status  and 
the  rate  of  change  during  live-fire  experience)  being  their  averages. 


Figure  19.  Estimated  change  trajectories  in  knowledge  maps  for  three 
hypothetical  2nd  LTs. 


The  process  of  learning  (the  change  in  knowledge  maps  over  time)  with 
intervening  instruction  or  practice  sessions  could  in  turn  provide  information  on  the 
sensitivity  of  knowledge  maps  as  a  measure  of  knowledge  to  instruction  or  practice 
in  the  domain  of  rifle  marksmanship.  As  explained  above,  on  average,  a  2nd  LT's 
knowledge  map  scores  increase  at  a  fast  and  significant  rate  (1.76  points  per  day) 
during  classroom  instruction,  but  there  is  significant  individual  variation  around  it. 
Thus,  it  may  suggest  that  knowledge  maps  tend  to  be  sensitive  to  classroom 
instruction  for  2nd  LTs,  but  that  the  extent  of  sensitivity  depends  on  the  individuals. 
It  would  be  interesting  to  further  examine  the  correlates  of  the  extent  of  sensitivity. 
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i.e.,  which  characteristics  of  the  2nd  LTs  relate  to  greater  sensitivity  to  classroom 
instruction. 

Also,  a  2nd  LT  still  shows  gains  in  his  knowledge  map  scores  during  live-fire 
exercises  after  classroom  instruction  (0.37  points  per  day),  although  the  rate  was 
much  slower  than  during  classroom  instruction.  This  may  also  suggest  that  the 
knowledge  map  measure  tends  to  be  sensitive  to  the  post-instruction  live-fire 
experience  for  2nd  LTs,  but  that  the  extent  of  sensitivity  depends  on  the  individuals. 

Knowledge  maps  of  the  current  sample  have  been  scored  in  more  than  one 
way.  The  same  statistical  model  has  been  applied  to  the  other  scores  of  knowledge 
maps.  The  patterns  of  change  are  fairly  similar  to  the  results  of  the  scores  analyzed 
in  this  section,  in  that  the  change  rate  was  high  and  significant  during  the  classroom 
instruction  period,  and  slower  but  still  significant  during  the  post-instruction  live- 
fire  experience  period,  and  also  in  that  there  was  significant  variation  in  the  growth 
pattern  across  individuals.  However,  the  level  of  scores  varied  substantially  between 
different  kinds  of  scores.  This  is  expected,  because  the  scoring  schemes  varied  in 
terms  of  leniency.  Consistency  across  different  types  of  scores  supports  the  results 
and  the  implications. 


